[hatari-devel] Code-review, unit tests and bug-tracker

Deniz Turkoglu turkoglu.deniz at gmail.com
Wed May 11 21:21:33 CEST 2011


On Tue, May 10, 2011 at 11:04 PM, Eero Tamminen <eerot at users.berlios.de> wrote:
> Hi,
>
> On tiistai 10 toukokuu 2011, Deniz Turkoglu wrote:
>> On Mon, May 9, 2011 at 10:41 PM, Eero Tamminen <eerot at users.berlios.de> >
>> On lauantai 07 toukokuu 2011, Deniz Turkoglu wrote:
>> >> I have been following the list for sometime and now I have the time to
>> >> contribute more, at least I hope so :) I have a few
>> >> questions/suggestions, mainly based on the flow we use at work;
>> >>
>> >> - We use ReviewBoard http://www.reviewboard.org/ at work. I think this
>> >> is nice for a few reasons. It makes discussing on changes easier and
>> >> you can associate changes with bugs and end results tends to be
>> >> better. I think this would speed up the discussion that goes on
>> >> patches?
>> >
>> > If BerliOS will support reviewboard, why not.
>>
>> It doesn't seem to have it.
>>
>> http://www.berlios.de/developers.php.en
>>
>> >> - I tried the bug tracker to have a go at things but seems it is not
>> >> as active as the mailing list, maybe we need another bug tracker, like
>> >> bugzilla or trac?
>
> Btw. why/how changing the bug tracker would help?
>
> E.g. bugzilla sends better bug mails and is otherwise nicer if everything
> would go through bug tracker, but stuff seems to work fine on devel list
> too.
>
>
>> > You can file bugs/tickets for BerliOS bug tracker itself too I assume.
>> >
>> > Or were you proposing on hosting reviewboard and bug tracking services
>> > for Hatari yourself?
>>
>> The bug tracker and reviewboard can be hosted elsewhere of course. I
>> have shared hosting and can give it a go, it is not a fancy hosting
>> company though, so maybe setting up a home server is a better option.
>
> I'd rather like BerliOS facilities to be enhanced...

+1

As I said it was just a suggestion to use the bug tracker more, for
instance as a TODO list, so when someone has time and wants to have a
look they can find something to work on.

>
> [...]
>> > Main bulk of Hatari testing is running different (hundreds of) Atari
>> > ST, STE and Falcon programs and just checking whether their output
>> > looks OK without (much) need to compare it to real HW though.  And
>> > checking that the output hasn't regressed.
>> >
>> > If you want to look into that, you should try "hconsole" python program
>> > in tools/hconsole/ and try the Hatari remote API on how you
>> > automate[1] running of the tests (ST/STE/Falcon demos).  Through that
>> > you could e.g. run demos and record sound output & get screenshots[1]
>> > from them.
>> >
>> > Comparing the sound files could be a bit inconvenient because 1-1 file
>> > compare probably isn't the best thing (you'd catch changes, not whether
>> > they're regressions).  I think for screenshots 1-1 checking should be
>> > fine for catching regressions though.
>> >
>> >
>> > [1]  Automating other things than demos would probably require adding
>> >    support for positioning Atari mouse on some specific co-ordinate
>> >    on Atari screen.
>> >
>> >    To get them screenshots etc at suitable time in the demo, adding
>> > some kind of "pause at VBL n from start/now" functionality to Hatari
>> > might be needed.  Or "dump screenshot at exit" coupled with
>> > "--run-vbls" option, if one screenshot per demo is enough.
>> >
>> >   Somehow detecting bus errors and other emulated Atari exceptions
>> >   through the remote API could also be useful.
>>
>> Perfect, I have done some work on this area, it turned out to be
>> harder than it looked at first sight :)
>>
>> Screenshots: we hash the buffer and do a compare on host (i.e: hashes
>> are sent to hatari and if it matches any it is a pass, else we
>> compress to png and send it over the wire.
>
> Screenshots don't go over the socket, they go to Hatari's current directory.
> Taking them is what you can trigger over remote API.
>
>
>> This is probably not very
>> useful for hatari as is since timing issues with frames are always a
>> problem, esp. considering different OS and devices. The idea to add
>> pause functionality sounds like a very good idea to solve this
>> bottleneck.
>>
>> Mouse/keyboard interaction: there are 2 options with this one, we can
>> either fake it within hatari or we can directly do it within OS level,
>
> Hatari already supports triggering key events and mouse button presses
> through the remote API.
>
> The thing lacking is moving the mouse, but that should mainly be needed just
> for GEM stuff.  And emulating GEM stuff shouldn't be a problem like
> emulating demos, so I think it can be mostly ignored...
>
>
>> both has its ups and downs, if we fake it we can not be sure if the
>> cursor really moved, was visible on screen etc.. If we do it natively
>> it requires to port to every single OS that we run on. Also one neat
>> part about doing it hatari level is that it can compile and run tests
>> and you can still work on the computer at the same time,
>
> It should also be more reliable (exactly the same events), host -> SDL ->
> Hatari event parsing timing issues cannot affect the movement events.
>
>
>> will have a look at this and how feasible it is.
>>
>> Audio: This one is rather tricky and I haven't done it before, 1-1
>> compare using the hash method can help with the regression, for
>> instance if we have a blacklist of hashes (regressions) we can find
>> out the root cause easier. Comments on this are more than welcome :)
>
> The screenshots & sound output are files saved by Hatari.  The control
> socket is a local unix socket, not a tcp/ip one.  I don't understand what
> you mean by "hashes"?

We hash the images (md5) and we keep a list of possible passes and
fails (regressions). In screenshots and audio files this can help us
to identify some things quickly (such as blank screenshot, empty audio
file, etc...).

>
>
>        - Eero
> _______________________________________________
> hatari-devel mailing list
> hatari-devel at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/hatari-devel
>



More information about the hatari-devel mailing list