[Lazarus] Stable islands: testing stability

[Chavoux Luyt]

+1

On 17 September 2012 07:39, Reinier Olislagers
<reinierolislagers at gmail.com>wrote:

> MML's post on stable islands has made me think.
>
> Below some ideas for discussion:
> If we want to know whether a certain combination of
> OS+FPC+Lazarus+packages is stable, we:
> 1. should ask the user
> 2. don't trust him completely though and we should get measurements
>
> Short-term action
> =================
> I suggest we can create a Lazarus GUI that
> - asks the user how stable he thinks Lazarus is
> - collects test measurement data (see below)
> The GUI can then display the results, save them for the user's own use
> (import in database etc), and upload to a central web page
>
> The GUI could run these tests:
> - compiler test suite. Custom test code; results available as XML, IIRC.
> - FPC database tests (e.g. for bufdataset, memds, sdfdataset, perhaps
> test Firebird embedded, sqlite, as well as any specified in the user's
> database.ini). These are fpcunit tests; can spit out e.g. XML
> - any other fpc tests written with fpcunit
> - Lazarus non-GUI tests (can't remember what data format these have)
>
> Medium-term action
> ==================
> - Have FPC report not only version but also SVN revision (much like
> lazarus does in help/about) so that data can be used to pinpoint exactly
> what version is used if checked out from SVN (e.g. trunk)
> - Write small cross platform database application to collect, read in
> test results into database; useful for user's own use
> - Similar but for a central web site. Web page with results could be
> done much like the compiler test suite page
> - Extend FPC fpcunit tests for packages (both in coverage and number of
> packages tested)
> - Extend Lazarus non-GUI tests (e.g. fill stringgrid, save as csv, load
> from csv, compare grid contents). Perhaps even a policy/behaviour
> change: no new units accepted without relevant tests (of course where
> feasible)
> - Extend packages so that there is a self-test possibility: the GUI
> mentioned above) and/or IDE shoul run unit tests in the packages and get
> the resutls. Example: Lazreport creates sample report, outputs to file,
> and compares file contents with expected contents
>
> Extending the test suites etc. would aid in regression testing as long
> as attention is paid to the trend in test faults.
>
> What do you think?
>
> Thanks,
> Reinier
>
> --
> _______________________________________________
> Lazarus mailing list
> Lazarus at lists.lazarus.freepascal.org
> http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20120917/68b719c8/attachment-0003.html>