[hatari-devel] Code-review, unit tests and bug-tracker

[Eero Tamminen]

Hi,

On tiistai 10 toukokuu 2011, Deniz Turkoglu wrote:
> On Mon, May 9, 2011 at 10:41 PM, Eero Tamminen <eerot at users.berlios.de> > 
> On lauantai 07 toukokuu 2011, Deniz Turkoglu wrote:
> >> I have been following the list for sometime and now I have the time to
> >> contribute more, at least I hope so :) I have a few
> >> questions/suggestions, mainly based on the flow we use at work;
> >> 
> >> - We use ReviewBoard http://www.reviewboard.org/ at work. I think this
> >> is nice for a few reasons. It makes discussing on changes easier and
> >> you can associate changes with bugs and end results tends to be
> >> better. I think this would speed up the discussion that goes on
> >> patches?
> > 
> > If BerliOS will support reviewboard, why not.
> 
> It doesn't seem to have it.
> 
> http://www.berlios.de/developers.php.en
> 
> >> - I tried the bug tracker to have a go at things but seems it is not
> >> as active as the mailing list, maybe we need another bug tracker, like
> >> bugzilla or trac?

Btw. why/how changing the bug tracker would help?

E.g. bugzilla sends better bug mails and is otherwise nicer if everything
would go through bug tracker, but stuff seems to work fine on devel list
too.


> > You can file bugs/tickets for BerliOS bug tracker itself too I assume.
> > 
> > Or were you proposing on hosting reviewboard and bug tracking services
> > for Hatari yourself?
> 
> The bug tracker and reviewboard can be hosted elsewhere of course. I
> have shared hosting and can give it a go, it is not a fancy hosting
> company though, so maybe setting up a home server is a better option.

I'd rather like BerliOS facilities to be enhanced...


[...]
> > Main bulk of Hatari testing is running different (hundreds of) Atari
> > ST, STE and Falcon programs and just checking whether their output
> > looks OK without (much) need to compare it to real HW though.  And
> > checking that the output hasn't regressed.
> > 
> > If you want to look into that, you should try "hconsole" python program
> > in tools/hconsole/ and try the Hatari remote API on how you
> > automate[1] running of the tests (ST/STE/Falcon demos).  Through that
> > you could e.g. run demos and record sound output & get screenshots[1]
> > from them.
> > 
> > Comparing the sound files could be a bit inconvenient because 1-1 file
> > compare probably isn't the best thing (you'd catch changes, not whether
> > they're regressions).  I think for screenshots 1-1 checking should be
> > fine for catching regressions though.
> > 
> > 
> > [1]  Automating other things than demos would probably require adding
> >    support for positioning Atari mouse on some specific co-ordinate
> >    on Atari screen.
> > 
> >    To get them screenshots etc at suitable time in the demo, adding
> > some kind of "pause at VBL n from start/now" functionality to Hatari
> > might be needed.  Or "dump screenshot at exit" coupled with
> > "--run-vbls" option, if one screenshot per demo is enough.
> > 
> >   Somehow detecting bus errors and other emulated Atari exceptions
> >   through the remote API could also be useful.
> 
> Perfect, I have done some work on this area, it turned out to be
> harder than it looked at first sight :)
> 
> Screenshots: we hash the buffer and do a compare on host (i.e: hashes
> are sent to hatari and if it matches any it is a pass, else we
> compress to png and send it over the wire.

Screenshots don't go over the socket, they go to Hatari's current directory.
Taking them is what you can trigger over remote API.


> This is probably not very
> useful for hatari as is since timing issues with frames are always a
> problem, esp. considering different OS and devices. The idea to add
> pause functionality sounds like a very good idea to solve this
> bottleneck.
> 
> Mouse/keyboard interaction: there are 2 options with this one, we can
> either fake it within hatari or we can directly do it within OS level,

Hatari already supports triggering key events and mouse button presses
through the remote API.

The thing lacking is moving the mouse, but that should mainly be needed just
for GEM stuff.  And emulating GEM stuff shouldn't be a problem like
emulating demos, so I think it can be mostly ignored...


> both has its ups and downs, if we fake it we can not be sure if the
> cursor really moved, was visible on screen etc.. If we do it natively
> it requires to port to every single OS that we run on. Also one neat
> part about doing it hatari level is that it can compile and run tests
> and you can still work on the computer at the same time,

It should also be more reliable (exactly the same events), host -> SDL -> 
Hatari event parsing timing issues cannot affect the movement events.


> will have a look at this and how feasible it is.
> 
> Audio: This one is rather tricky and I haven't done it before, 1-1
> compare using the hash method can help with the regression, for
> instance if we have a blacklist of hashes (regressions) we can find
> out the root cause easier. Comments on this are more than welcome :)

The screenshots & sound output are files saved by Hatari.  The control
socket is a local unix socket, not a tcp/ip one.  I don't understand what
you mean by "hashes"?


	- Eero