# reportfmt

title: All Tickets
cols:
  #ffffff Key:
  #f2dcdc Active
  #e8e8e8 Review
  #cfe8bd Fixed
  #bde5d6 Tested
  #cacae5 Deferred
  #c8c8c8 Closed

sqlcode:
  SELECT
    CASE WHEN status IN ('Open','Verified') THEN '#f2dcdc'
         WHEN status='Review' THEN '#e8e8e8'
         WHEN status='Fixed' THEN '#cfe8bd'
         WHEN status='Tested' THEN '#bde5d6'
         WHEN status='Deferred' THEN '#cacae5'
         ELSE '#c8c8c8' END AS 'bgcolor',
    substr(tkt_uuid,1,10) AS '#',
    datetime(tkt_mtime) AS 'mtime',
    type,
    status,
    subsystem,
    title
  FROM ticket


# event

type: ci
user: aku
comment:
  Fixed missing cleanup of tmp directories. Made the lock less unique to actually catch simultaneous operation.


type: ci
user: andreask
comment:
  More work on reports.
  Reworked the handling of template default, reorganized the command set: 'show', 'set', 'reset' now.
  Started working on 'import, 'edit'.
  Tweaked 'export'.
  Refactored internals a bit, around 'export' and other places retrieving report information.


type: ci
user: andreask
comment:
  Bugfixes in 'list' command.
  New command 'show' for details of particular report.
  New command 'copy' to copy report definitions.
  New command 'set-owner' to change report ownership.


type: ci
user: andreask
comment:
  Changed mail for commits, limit amount of paths shown per action (max 5, limit fixed) to prevent mail size from getting overly large.


type: t
user: aku
brief: Ticket [17905ce5ad]: 3 changes
comment:
  Ticket [17905ce5ad] <i>Implement easy access to ticket non-report setup</i> status still Open with 3 other changes


type: g
user: aku
comment:
  Edit [3c079f6318d50a88]: Move to branch [/timeline?r=mistake&nd&dp=3c079f6318d50a88&unhide | mistake]...


type: t
user: aku
brief: Ticket [b23f9a08af]: 4 changes
comment:
  Ticket [b23f9a08af] <i>Fix the 'open' command for all cli &quot;export&quot; methods</i> status still Open with 4 other changes


type: ci
user: aku
comment:
  Fixed formatting of the Date header in generated mail to be RFC-2822 compliant, per the report by Schelte Bron.


type: t
user: aku
brief: Ticket [97ebd0c480]: 3 changes
comment:
  Ticket [97ebd0c480] <i>Implement easy access to ticket reports</i> status still Open with 3 other changes


type: t
user: aku
brief: Closed ticket [b23f9a08af].
comment:
  Closed ticket [b23f9a08af]: <i>Fix the 'open' command for all cli &quot;export&quot; methods</i> plus 2 other changes


type: t
user: aku
brief: Ticket [192e316cf7]: 4 changes
comment:
  Ticket [192e316cf7] <i>Auto-assignment of tickets to dev based on categorization</i> status still Open with 4 other changes


type: ci
user: andreask
comment:
  Added fx::atexit package for setting of exit handlers.
  Extended the low-level fossil package to use exit-handlers for proper
  DB shutdown on exit.


type: ci
user: andreask
comment:
  Removed debug code from "test mail-address", and extended to handle semicolon-separated lists as well.


type: t
user: aku
brief: Ticket [ebb5370ec1]: 5 changes
comment:
  Ticket [ebb5370ec1] <i>pseudo mail storm when failing mail</i> status still Open with 5 other changes


type: t
user: aku
brief: Ticket [ebb5370ec1]: 3 changes
comment:
  Ticket [ebb5370ec1] <i>pseudo mail storm when failing mail</i> status still Open with 3 other changes


type: ci
user: andreask
comment:
  Ticket [b23f9a08af] - Added utility command creating directory for
  file as part of opening it. Updated all export backends to use this
  utility command instead of plain open. Still to test.


type: ci
user: andreask
comment:
  Reworked notification delivery to handle mail errors better. Reduce pseudo-storm to entries which fail and still generate a mail. More context in fx error mails.


type: ci
user: andreask
comment:
  fx::peer - Ticket [f134299cf7] - Added code to detect attampts at
  sharing a .peer-state directory between different projects, and block
  them.


type: t
user: aku
brief: Ticket [f134299cf7]: 4 changes
comment:
  Ticket [f134299cf7] <i>Detect attempts to share state directories between repositories</i> status still Open with 4 other changes


type: ci
user: andreask
comment:
  Various new functionality.
  fx::map  - Added a command 'map rename'.
  fx::enum - Added a command 'enum rename'.
  fx::peer - Added commands 'peer export|import'. Internal factoring.
  Testsuites extended for the new commands, incomplete on expected results.
  Removed a number of bogus TODO notes.


type: ci
user: andreask
comment:
  The fx tool has subsumed all features of the old mirror and watch scripts (plus more). Time to make it official, merged into trunk.


type: ci
user: andreask
comment:
  fx::peer - Added advanced ops to reset/clear git state, factored some checks (state? owned?) into separate commands.


type: t
user: aku
brief: Ticket [b23f9a08af]: 3 changes
comment:
  Ticket [b23f9a08af] <i>Fix the 'open' command for all cli &quot;export&quot; methods</i> status still Open with 3 other changes


type: t
user: aku
brief: Closed ticket [7a2b439c47].
comment:
  Closed ticket [7a2b439c47]: <i>do-mirror failures can leave system in state where it will not mirror until actual new change happens.</i> plus 4 other changes


type: t
user: aku
brief: Closed ticket [f134299cf7].
comment:
  Closed ticket [f134299cf7]: <i>Detect attempts to share state directories between repositories</i> plus 4 other changes


type: ci
user: aku
comment:
  fx::peer - Tweaked the log output generated during exchange. Shuffled a few things around for clarity.


type: t
user: aku
brief: Ticket [ab81364628]: 5 changes
comment:
  Ticket [ab81364628] <i>setup-export fails with some git versions</i> status still Open with 5 other changes


type: t
user: aku
brief: Ticket [7a2b439c47]: 4 changes
comment:
  Ticket [7a2b439c47] <i>do-mirror failures can leave system in state where it will not mirror until actual new change happens.</i> status still Open with 4 other changes


type: ci
user: aku
comment:
  fx::peer, fx::validate::not-peer-git|fossil - Ensure initialization in v-types. Fix various issues in the add|remove commands.


type: ci
user: aku
comment: fx::fossil - Maps & enumerations. Fix sql code to allow special characters in table names.
ecomment:
  fx::fossil - Maps & enumerations. Fixed low-level sql code to allow special characters in table names.


type: t
user: aku
brief: Ticket [7a2b439c47]: 5 changes
comment:
  Ticket [7a2b439c47] <i>do-mirror failures can leave system in state where it will not mirror until actual new change happens.</i> status still Open with 5 other changes


type: t
user: aku
brief: Ticket [17905ce5ad]: 4 changes
comment:
  Ticket [17905ce5ad] <i>Implement easy access to ticket non-report setup</i> status still Open with 4 other changes


type: t
user: aku
brief: Closed ticket [e107d42e07].
comment:
  Closed ticket [e107d42e07]: <i>Add &quot;suspension&quot; flag for notification delivery.</i> plus 4 other changes


type: ci
user: andreask
ecomment:
  fx::tty, fx::term, fx::color - Replaced with the equivalent new cmdr packages, callers updated, removed.

comment:
  fx::tty, fx::term, fx::clor - Replaced with use of equivalent new cmdr packages, callers updated, removed.


type: t
user: aku
brief: Ticket [7a7612bf2b]: 4 changes
comment:
  Ticket [7a7612bf2b] <i>Need key-dependent validation of values</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [56f4456ac2]: 4 changes
comment:
  Ticket [56f4456ac2] <i>Make field truncation in mail generation configurable</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [ef1ed37c8e]: 5 changes
comment:
  Ticket [ef1ed37c8e] <i>Make generation of mails configurable via templates</i> status still Open with 5 other changes


type: ci
user: aku
comment:
  fx - Added highlevel spec of the map management commands. Todo: Backend bridging to the low-level API.


type: t
user: aku
brief: Ticket [e107d42e07]: 4 changes
comment:
  Ticket [e107d42e07] <i>Add &quot;suspension&quot; flag for notification delivery.</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [b7032cb56f]: 4 changes
comment:
  Ticket [b7032cb56f] <i>For some config keys --global makes no sense</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [1e5e7b0c6d]: 5 changes
comment:
  Ticket [1e5e7b0c6d] <i>Make processing of ticket field values configurable</i> status still Open with 5 other changes


type: t
user: aku
brief: Ticket [ef1ed37c8e]: 3 changes
comment:
  Ticket [ef1ed37c8e] <i>Make generation of mails configurable via templates</i> status still Open with 3 other changes


type: t
user: aku
brief: Ticket [aed2152013]: 4 changes
comment:
  Ticket [aed2152013] <i>Make the colorization configurable.</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [23724e0344]: 4 changes
comment:
  Ticket [23724e0344] <i>Make cron_lock more robust on unix.</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [55d0b9e7c6]: 4 changes
comment:
  Ticket [55d0b9e7c6] <i>Design and implement the mirroring support</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [39af71d253]: 3 changes
comment:
  Ticket [39af71d253] <i>Make #retries (and delay) for get-manifest configurable</i> status still Open with 3 other changes


type: t
user: aku
brief: Ticket [b6a9ba9bae]: 4 changes
comment:
  Ticket [b6a9ba9bae] <i>Extend the notification system for additional transmission channels</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [ab81364628]: 4 changes
comment:
  Ticket [ab81364628] <i>setup-export fails with some git versions</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [1e5e7b0c6d]: 3 changes
comment:
  Ticket [1e5e7b0c6d] <i>Make processing of ticket field values configurable</i> status still Open with 3 other changes


type: t
user: aku
brief: Ticket [65de12c710]: 4 changes
comment:
  Ticket [65de12c710] <i>Add command history support to the app</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [39af71d253]: 6 changes
comment:
  Ticket [39af71d253] <i>Make #retries (and delay) for get-manifest configurable</i> status still Open with 6 other changes


type: t
user: aku
brief: Ticket [97ebd0c480]: 4 changes
comment:
  Ticket [97ebd0c480] <i>Implement easy access to ticket reports</i> status still Open with 4 other changes


type: t
user: aku
brief: Ticket [7e91e835c7]: 4 changes
comment:
  Ticket [7e91e835c7] <i>Add ability to exclude or map specific mail destinations.</i> status still Open with 4 other changes


type: g
user: aku
comment:
  Edit [61a03a94b6a2d461]: Move to branch [/timeline?r=fx-peers&nd&dp=61a03a94b6a2d461&unhide | fx-peers]...


type: t
user: aku
brief: Ticket [aed2152013]: 3 changes
comment:
  Ticket [aed2152013] <i>Make the colorization configurable.</i> status still Open with 3 other changes


type: ci
user: andreask
comment:
  fx - Modified to request a "cmdr::actor" version with "-extend" support, and use it to thread the global --debug/--color specs into subordinates declaring their general options (--repository usually).


type: ci
user: aku
comment:
  fx::seen, fx::mgr::state - Fixed handling of seen uuids, cannot save/restore rids, have to go through uuids in the state file.


type: ci
user: andreask
comment:
  fx::note - Added code to prevent mail storms, checking the number of pending changes against the configured limit. When triggered a mail is send to the admin ("sender" config)


type: ci
user: andreask
comment:
  fx::validate::mail-config - Added a "limit" option to detect and prevent mail storms. Default limit is 10.


type: ci
user: andreask
comment:
  Installation - Reactivate install of "cron_lock" helper, and fully drop the old set of watch-* apps.


type: ci
user: andreask
comment:
  fx::shun- Implemented add/remove, fx::term - New support package for simple interactions, fx::color - Additional symbolic color


type: ci
user: andreask
comment:
  fx, fx::shun - New commands to manage the list of shunned artifacts. Incomplete. So far we can only show the list.


type: ci
user: aku
comment:
  Modified the calculation of receivers: login field => entire history of the field. user making the change => drop, do not notify.


type: ci
user: andreask
comment:
  fx::validate::uuid, fx::shun - Extend list to show if content still available or not. Outline of the add/remove commands.


type: ci
user: aku
comment:
  fx::mgr::state, fx::state, fx::seen, fx::enum, fx::mgr::enum - Fixed lots of buglets in the save/restore code.


type: ci
user: aku
comment:
  fx::mailgen - Fixed unwanted truncation of ecomment for mail body. Limit the truncation to the subject line.


type: ci
user: andreask
comment: fx::seen - Added iterator for the notified events.
ecomment:
  fx::seen - Added iterator for the notified events.
  fx::note - Added listing of notified events (vs pending), and extended both to allow display of extended type information.
  fx - Added the "note show-notified" command.


type: ci
user: andreask
comment:
  cron_lock - Added meta data. Note: Found out about "flock", which is better by far. May remove cron_lock in the future.


type: ci
user: andreask
comment:
  Converted last example. Shuffled the file names a bit. Still have to diff the various blocks for unseen commonalities.


type: ci
user: aku
comment:
  Updated and extended testsuite to match the last round of changes. Fixed bugs uncovered by the work. Some extensions: "header" attribute for mails.


type: ci
user: andreask
comment:
  fx::enum - Modified to use the fx::mgr::enum now. fx::mgr::enum - Impose proper ordering (order of item creation).


type: ci
user: andreask
comment: fx::validate::enum - Created helper command "table-of" for name -> table conversion.
ecomment:
  fx::validate::enum
  - Created helper command "table-of" for name -> table conversion.

  fx::enum
  - Reworked to use the standard fossil support commands.
  - Colorization
  - Debug narrative
  - Regular user messaging

  fx::note
  - Slight reordering of requirements.
  - Added missing debug requirements.


type: ci
user: andreask
comment:
  fx::mgr::enum - New package. Low-level enum operations. No high-level validations => Can use this for fx-internal enums to store things.


type: ci
user: andreask
comment:
  fx::note, fx::mailgen - Added proper footer to the test mail for the mail-config. Dropped wrongly introduced mailer dependency in the generator.


type: ci
user: andreask
comment:
  Reworked the mail config export for global/local/mix (latter is default, old mode). Added reset command. Extended show command like export.


type: ci
user: andreask
comment:
  Moved all titling into the definition file and replaced all references in the docs with the proper variables.


type: ci
user: andreask
comment:
  fx::state, fx::mgr::state - Extended the foundation with helper commands for easy dumping of any table.


type: ci
user: andreask
comment:
  fx::mailer - New method for access to all parts of the mail config. Plus pseudo-key for project name. Dropped sender-specific method, not used anyway.


type: ci
user: andreask
comment:
  Reworked use of option --repository|-R, made explicit instead of *all*, and different mechanism to disable for operations whih have a global mode.


type: ci
user: andreask
comment:
  cli, fx::enum, fx::note, fx::state - Modified export/import API. More consistent parameter names, and consistent parameter types (input).


type: ci
user: aku
comment:
  Reordered mail generation logic for future extensibility (Failure at bottom, positive guards into generation, change type)


type: ci
user: andreask
comment:
  fx::seen - Moved another bit of code into a helper procedure. Renamed the regeneration command, and added a --clear option, changing the default behaviour to incremental


type: ci
user: aku
comment:
  fx::note - Adding handling of extended event-type "all" to the route add/drop commands. Plus colorization.


type: ci
user: aku
comment:
  fx::note - Fixed argument type mismatch, and added automatic dynamic receiver, the account which made the change.


type: ci
user: aku
comment:
  fx::seen - Added early aborts for history when no fields, or no pending changes. Fixed issue with set-watched-fields leaving an empty entry behind when setting the entire set to empty.


type: ci
user: aku
comment:
  Expanded the "repository" command into a hierarchy, and added management of a default repository, so that we can avoid the use of lots of -R ... in fx commands. Modified most commands to show the work repository.


type: ci
user: andreask
comment:
  fx::seen - Refactored the insides of FillSeries to make the code more readable, and as prep of a rwrite for chunked transactions.


type: ci
user: andreask
comment:
  fx::fossil - Reworked get-manifest to detect lock-ups and hopefully deal with them in a few retries.


type: ci
user: andreask
comment:
  fx::seen - Rewritten to chunked transactions with restart after a lock-out if we had made some progress.


type: ci
user: aku
comment:
  fx::manifest - Removed all uses of "regexp" from the parser, replaced with "split". Much faster, especially on very large arguments (hundreds of KB).


type: ci
user: aku
comment:
  Implemented option --(no-)color, and command "note watched" listing the repositories with active routes. Moved the config keys into variables for easier maintenance. Reworked the config access, make use of the new config and utility commands to simplify the code. Tweaks to the output formatting.


type: ci
user: andreask
comment:
  Write up of a basic use-case: Remote monitoring of a single repository, as repo admin, but not host admin.


type: ci
user: aku
comment:
  fx::mailgen - General tweaks to output (closing empty line), and completion of control notification (Proper link to changed checkin|event)


type: ci
user: aku
comment:
  fx::validate::event-type - Added type labeling for vertical alignment. Plus extended event-type VT including a special value "all".


type: ci
user: aku
comment:
  Added helpers to determine number of pending things. Tweaked progress reporting on fill-series. More places to run fill-series.


type: ci
user: aku
comment:
  Modified validation type for enum names, disallowed a number of special characters. Allows for the definition of fx internal enums a user cannot create. Example: A delivery exclusion list.


type: ci
user: andreask
comment:
  Added some basic colorization (trapped errors), and started on adding a debug narrative with cli support.


type: ci
user: aku
comment:
  Extended fx::mailer, added helper command to deduplicate mail addresses in the presence of human readable parts. In that case the longest address in each class is kept.


type: ci
user: aku
comment:
  Many changes to the fx cmdr spec, and the note hierarchy implementation
  - Added import/export commands for mail config, and routes.
  - Resolved a number of TODO's regarding missing cmdr::parameters' needed for internal validation tasks.
  - Added the calls to manage the ticket field timeseries cache.
  -- Including a public command to force cache regeneration
  - Completed the calculation of dynamic receivers from ticket information.
  - Extended set of exceptions caught.
  - Reworked delegation to fossil for easy setup, added catching of errors from fossil, these are not fx errors.
  - Started on a 'user' command hierarchy (No backend yet).
  - Many commands now show the repository file they are working with as first message of the output.

  TODO
  - Mail exclusion
  - User management (contact information).
  - Extend the test commands for manifest-parse, mail-for, mail-receivers
    to allow operation, and thus testing, of all events in a repository


type: ci
user: aku
comment:
  Added --all support to test-parse. Fixed extype use in the other test-* commands. Restricted test-parse and -mail-gen to show only fail cases. Changed test-parse output to show a proper table. Switched to new mail deduplicator command. Modified project-info retrieval to use the new location default if the field is not set.


type: ci
user: andreask
comment:
  Modified commands "config unset" and "note config unset" to accept multiple keys for batch deletion.


type: ci
user: aku
comment:
  Started on the test commands accepting --all. Exclusion with uuid, and check that one must be set. Refactored receiver testing for this, and sharing with deliver.


type: ci
user: aku
comment:
  Reworked the parser a bit, may have cards with key, but no argument => no space after key. Moved space handling out of the regexp biting cards from the front of the string, into the argument handling code (regexes, trim). Detected because of mis-parsed initial commit having wrong type and generator assert.


type: ci
user: aku
comment:
  fx::validate::mail-config - Special cases the default for location to use the last-sync-url, if not locally overridden. Less work for tracking with a mirror.


type: ci
user: aku
comment:
  fx::user, and related: Extended list to show change time and notes (no email). Completed implementation of contact command. Added mail broadcast command, and shortcuts to push/pull/sync the user config.


type: ci
user: aku
comment:
  Merged the mark-* commands pairs into a single command allowing uuid and --all. Added debug narrative.


type: ci
user: aku
comment:
  Fixed bug left over from the parser change, field names in ticket changes got a bogus leading space.


type: ci
user: andreask
comment:
  Added lexical email address checking. Moved the validation of imported data into the hidden state parameters instead of calling it on our own and using them only as a source of labeling any generated error messages.


type: ci
user: aku
ecomment:
  Updated notes regarding use of current ticket state. Superceded by the new ticket timeseries cache in fx::seen

comment:
  Updated notes regarding use of current ticket state. Supeperceded by the new ticket timeseries cache in fx::seen


type: ci
user: aku
ecomment:
  Extended the management of seen events to maintain a timeseries cache for the ticket fields of interest, i.e. used to generate the dynamic mail destinations.

comment:
  Extended m the management of seen events to maintain a timeseries cache for the ticket fields of interest, i.e. used to generate the dynamic mail destinations.


type: ci
user: aku
ecomment:
  Extended the low-level repository access with helper commands to transform strings via concealed and user information.

comment:
  Extended the low-level repository access w with helper commands to transform strings via concealed and user information.


type: ci
user: aku
comment:
  Jiggered the event type information, and the ecommand, aka timeline text to match the actual texts seen in the origin repositories of the examples


type: ci
user: andreask
comment:
  Near completed mail for control artifacts. Refactored all type of links into central generation command all others use.


type: ci
user: aku
comment:
  Test support, fixed sql syntax for the strings, handle quoting properly and prep to force app to ignore the terminal width.


type: ci
user: aku
comment:
  Fleshed out the mail generator code. Added tests and expected test results. The generation for control artifacts is incomplete, has to recurse into referenced manifests to infer type information.


type: ci
user: aku
comment:
  Sorted out the low-level method naming, naming in the action layer, and command specification. Updated help results to match. Back in low-level layer "seen", fixed pending-all, must not drop the table, just empty it, otherwise forall-pending will break.


type: ci
user: aku
comment:
  Added tests for the control methods. Passing. Now we can start testing the next higher layers, i.e. manifest parsing, then mail generation, and receiver collection.


type: ci
user: aku
comment:
  Fix scope issue in the info link construction in the generator. Plus reworked handling of the sender address, now inserted via the smtp configuration instead of the generated mail. Fixes issue with multiple from addresses, some internal.


type: ci
user: aku
comment:
  Fixes to example manifest storage and use. Save event type and comment with them. Explicitly add them to the event table, rebuild will not do that for us, with the missing references.


type: ci
user: aku
comment:
  Modified show-pending to adapt to terminal width, with override by environment. Fixed some typos. Extended test commands to determine and use context information (timeline data) for manifests.


type: ci
user: aku
comment:
  Split the manifest example archive into separate files. Use masking header in each. Fixes issues with setting the examples up for use. With separate files it works and no issues with splitting. Updated the test for setup. Now working


type: ci
user: andreask
comment:
  Reworked parser to handle W card properly. Forces manual extraction of lines from the front. Consider use -start and -indices for the regexes to avoid copying string data.


type: ci
user: aku
comment:
  Fixed main command export.
  Fixed return code tricksiness for the global database.
  Fixed wrong command name for repository database.
  Fixed handling of relative path to repository from checkout (Must be relative to the checkout).
  Tweaked error message for missing repository db, fixed access to global variable.
  Normalize location of global database to better see actual path in debugging.


type: ci
user: andreask
comment:
  Moved manifest parser to separate package, extended to handle all possible types. Integrated to notification. Project name/location handling built into the mail part.


type: ci
user: aku
comment:
  Testsuite fixes. Prevent interference of checkout and temp fossil repository used for testing. Use installed fx app for testing (Easier for now vs. trying to restrict to test-installs of the packages).


type: ci
user: aku
comment:
  Fixed main command export.
  Fixed import of fossil support.
  Fixed consistent namespace typo in all proc definitions.


type: ci
user: aku
comment:
  Updated manifest parser notes, and code tweaks. Plus file holding several example artifacts usable in testing.


type: ci
user: andreask
comment:
  Added draft test commands to help testing and debugging of the various aspects of notification generation and delivery.


type: ci
user: andreask
comment:
  Added basic mail generator code lifted from watch-do. Filled in mail delivery code using the new backend functionality. Missing, still: Dynamic field resolution. General suspension, account suspension, configurable title field, configurable ticket field action. Extension beyond tickets. Debug narrative. Dry runs. Route import/export. Test commands.


type: ci
user: andreask
comment:
  Half-way implementation of the notification: Mgmt commands complete, delivery missing. Plus extensions in the helper packages to support the previous. Tweaks to the test support.


type: ci
user: andreask
ecomment:
  Enumerations - vt separation, and completed implementation. Cmdline API changes allowing multiple enums/items in various places.

comment:
  Enumeratoins - vt separation, and completed implementation. Cmdline API changes allowing multiple enums/items in various places.


type: ci
user: aku
comment:
  Config: Fixes to "set" and "list" commands. New command "unset". General: Defaults specified. New command "repository" (introspection).


type: ci
user: aku
comment:
  Rework system into single app "fx". Use cmdr framework. Store as much as possible in extension tables of the various fossil databases.


type: ci
user: andreask
comment:
  Tweaked the report API (note: no implementation yet). Added notes about possible future functionality


type: ci
user: andreask
comment:
  More work on the notification functionality, command API, parts of the backend, using the new config mgr.


type: ci
user: andreask
comment:
  Moved the main application functionality into a package, reducing the actual application to, in essence, "package require fx ; fx do $argv"


type: ci
user: aku
comment:
  Extended watch-do with flag controlling its behaviour: disable saving of seen artifacts, disable generation of the notification mails.


type: ci
user: aku
comment:
  Moved the hardwired value of 200 for retrieval of timeline RSS entries
  to a configuration value ('num-entries'), with default of 200.

  Extended the database schema of table 'seen' with a column 'atime' to
  hold the last time the artifact was seen in the timeline RSS (epoch).

  Modified the database schema of table 'seen', going from the constraint
  	unique (pid)
  to the constraint
  	unique (pid,hash)
  => Guard against the possibility of artifact ids not unique across the
     entirety of all repositories, but only within a single repository.

  Added a configuration value ('days-to-expiry') for the expiration
  threshold (in days), with a default value of 30.

  Created command 'watch-schema-update-002-seen-expiry' to update the
  database schema of existing state databases.

  Modified 'watch-do' to use the new configuration value 'num-entries'.

  Modified 'watch-do' to touch the atime of seen artifacts as means of
  resetting their expiry timeout.

  Modified 'watch-do' to extract the ticket titles from the timeline RSS
  entries, for use in the generated mail, if any. As a note on that, the
  thought that we would need a clone of the repository to get this
  information was obviously wrong.

  Created command 'watch-expire' to scan the state database for expired
  artifacts, and remove them.

  Created command 'watch-cron' as the entry point for cron, calling do
  and expire, with locking.


type: ci
user: aku
comment:
  Added command to unsee an artifact, forcing regeneration of mail the next time watch-do is run. If it was a ticket change.


type: ci
user: aku
comment:
  Moved fixed mail headers into the mail generation, and tweaked the mail date to a possibly more portable form. Have a reader not understanding the iso format.


type: ci
user: andreask
comment:
  Schema change: configuration data moved to shared state database. Three new commands to query and modify the configuration. Factored db initialization into same procedure in all commands (execpt setup). Lastly, new script to perform the schema update for existing state databases


# ticket

tkt_uuid: 7a2b439c472cc9b51457f49dcd2782405850a8b4
type: Code_Defect
status: Closed
priority: Immediate
severity: Critical
foundin: 5d2d48d66972
private_contact: 4ca203a8e925aecdd9419645f936bccf18cd6ea7
resolution: Fixed
title:
  do-mirror failures can leave system in state where it will not mirror until actual new change happens.


# ticketchng

login: aku
mimetype: text/x-fossil-plain
icomment:
  The file format used by "fossil config export" is geared towards easy saving and restoring of a repository's configuration.

  As such it has some features which make it difficult to edit manually (char length information for cards) and thus not very good for non-web editing and modification of the ticket system. 

  Implement better programmatic access to the ticket pages and code in a form which allows for easy manual editing.

  This is a companion to the ticket [97ebd0c480] about easy access to the ticket reports.


login: aku
mimetype: text/x-fossil-plain
icomment:
  General ticketing components

  - table definition (TICKET, TICKETCHNG)
  - timeline display
    - name of title column (actual sql expr)
    - name of human readable status column
    - sql expression which is true for a "closed" ticket
  - TH1 code
    - common
    - run after ticket creation/editing
  - HTML pages (with TH1)
    - new ticket
    - view ticket
    - edit ticket
    - report list
  - report template (sql) => possibly better under [97ebd0c480] 
  - color key template    => ditto.

  Interesting features
  - Mapping enums and maps into TH1 common (dicts and lists)
  - Mapping the user table into TH1 common, via some shadow table (to avoid the authorizer on the regular user table).


login: aku
mimetype: text/x-fossil-plain
icomment:
  The open command does not take into account that the directory for the file may not exist. The "wfile" validation type allows this.


login: aku
mimetype: text/x-fossil-plain
icomment:
  With two or more notifications pending, the following can generate a pseudo-mail storm (*):
  - First note is mailed successfully
  - Second or later note fails

  The failure rolls the transaction back, leaving all pending, even the ones which were successful. On the next cycle mail is sent for them again.

  The only good thing in this situation is that fx has the feature of catching internal errors and sending them (their stacktrace) to the maintainer of a setup. I.e. essentially nagging the maintainer about the same pseudo-storm the users are seeing.

  (*) It is only a pseudo-storm because notifications are cron-based and as such do not happen in a rapid burst, but only every few minutes.

  Any fix has to stop the pseudo-storm to users, and should retain the pseudo-storm to the maintainer.

  This could be done by catching the issue within the transaction, preventing the rollback. But also still sending the error mail from the same place.

  This leaves the entries breaking mail unsent while the others are marked properly and properly drop out of the notification queue.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Definitely something to fix immediately.
  And then look into what caused the mail failure for Tcllib.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Possibly a size limitation ...
  A mail was actually sent despite fx getting an abort/error.
  It was partial.
  It had a very high number of lines (added files).
  Fix for that ... Show only the first 5 files in each section (add|dele|change), plus truncation note.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Mitigation done with [f94c3cb1cc].
  Failure to mail a notification does not stop processing of other pending notifications.
  This restricts the failure "storm" to the problematic changes.
  It also prevents the accumulation of more pending mails while waiting to resolve the issue.

  Semi-fix with [2984bfa396].
  Prevents generation of overly large mails, which seems to have been the issue.

  Merged with [7aadb9e42f].
  Closing the issue.
  If new causes for mail trouble appear we can fix them through new tickets.


login: aku
mimetype: text/x-fossil-plain
icomment:
  When a user submits a ticket they usually provide information like severity, type of ticket, and (dependending on repository configuration) things like functional area, etc.

  Create a system which can inspect and modify tickets based on field values.
  For example, assign the ticket to a developer based on the functional area, categorization, etc.

  A possible implementation would be as a collection of rules and actions, where rules match against ticket fields, and actions modify fields.

  The actual changes should be done through the "fossil ticket" command hierarchy to ensure that proper ticket changes are made, i.e. cause notifications to be sent, as per the "fx note" setup.

  Note: Rules must be ordered. We have to ensure that a fallback matches only after all other rules.

  Rules are sequences of and-linked clauses.
  A clause is a triple (field, operation, pattern) where the operation specifies how to match pattern against field value. To consider: substring, regexp, glob, exact.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The file format used by "fossil config export" is geared towards easy saving and restoring of a repository's configuration.

  As such it has some features which make it difficult to edit manually (char length information for cards) and thus not very good for non-web editing and modification of the ticket system. 

  Implement the "report" hierarchy with better programmatic access to reports in a form which allows for easy manual editing.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Report components:

  - name (aka title)
  - sql (select statement)              -- should have a template to use
  - color key (map: #rrggbb => string)  -- should have a template

  Note ! The color key is optional

  Report operations

  - List (table, raw, json)
  - Create name sql
  - Change name -q sql -c colors
  - Delete name
  - Export name (not multiple) - Restricted export, sql or color
  - Import file
  - Run name
  - Run <sql>|<sqlfile>  (create + run + delete, tmp-name)
  - Edit name   => auto-calls editor, (export + edit + import, tmp-file)
  - Edit restricted => sql, or colors


login: aku
mimetype: text/x-fossil-plain
icomment:
  It is very bad to point multiple fossil repositories to the same state directory. The files and directories in the state directory have fixed names, causing the fossil repositories to fight over they contents.

  Despite this the system currently does not recognize the situation.
  Change that, add code to
  (a) claim a state a directory (=> save project-code)
  (b) detect claims by other projects (existing project code found != ours).


login: aku
mimetype: text/x-fossil-plain
icomment:
  fx has subsumed the watch-* script of the trunk.
  The only feature of the trunk it does not support are the handling of mirrors (setup-*, do-mirror), in particular the management of mirroring to git.

  We must implement this in fx for feature parity and merging this branch back into the trunk.


login: anonymous
mimetype: text/x-fossil-plain
icomment:
  With some git versions (at least for git 1.7.3), setup-export fails with an error.

  In proc do setup-export does 

      run git --bare --git-dir=$git init
      file rename \
  	$git/hooks/post-update.sample \
  	$git/hooks/post-update

  Some git versions (or maybe git with some global settings) already create a post-update sh file (but without the execution flag of post-update.sample). Therefore, the [file rename] after the git init fail because the file post-update exists. A fix seems simple:

  Index: bin/setup-import
  ==================================================================
  --- bin/setup-import
  +++ bin/setup-import
  @@ -38,11 +38,11 @@

       set ::env(TZ) UTC

       puts "\tSetting up $git ..."
       run git --bare --git-dir=$git init
  -    file rename \
  +    file rename -force \
          $git/hooks/post-update.sample \
          $git/hooks/post-update

       fileutil::touch     $git/git-daemon-export-ok
       fileutil::writeFile $git/description \


login: aku
mimetype: text/x-fossil-plain
icomment:
  Merged with revision [f3ffa72fe5], for details see branch [fx-peers].
  Will not be fixed for the old scripts.


login: anonymous
mimetype: text/x-fossil-plain
icomment:
  fossil2git aborted a do-mirror (because of not being able to push to the git repo because of a mistyped password), but left the <state> dir in a state signaling, everything was mirrored succesful. 

  A second do-mirror run said "no new commits" already after updating the fossil repo to sync (which is correct, but the new commits from the previous do-mirror run aren't synced to the git repo).


login: aku
mimetype: text/x-fossil-plain
icomment:
  We either have to store the last-pushed-uuid on a per-remote basis, or delay saving the last-(pulled-)uuid until all pushes have been successful.

  In tend towards separating the last-(pulled-)uuid from the last-pushed-uuid-$remote and updating each piece as things succeed.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Extend fx to support management of named mappings, from key strings to key values. A natural extension of the enumerations already supported.

  A number of other tickets can be naturally implemented via such mappings, i.e.:
  * [1e5e7b0c6d] - ticket field action (field name -> action, label)
  * [aed2152013] - colorization maps (symbolic -> color, color -> definition)
  * [ef1ed37c8e] - mail templates (type -> template)
  * [7e91e835c7] - destination map (email -> email|nothing||notes)

  (For these the mappings would be internal and not visible to the commands for general user-defined mappings).


login: aku
mimetype: text/x-fossil-plain
icomment:
  The ticket
    [55d0b9e7c6] (mirroring support)
  can also use mappings as a fundamental data structure.
  Bumping priority, and implementing this first.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Done
  * [860005c7c7] Merged
  * [b438756903] Bugfixes, completed testsuite.
  * [0a99f589c6] Small fix to map tests
  * [2439423f26] Draft map management command implementations, draft testsuite.
  * [3e614fbc41] fx - Added highlevel spec of the map management commands.
  * [60e859c5b9] Implemented low-level map manipulation.
  * [f1d238f3cc] Implemented low-level map management, and validation types.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Currently the only way to temporarily disable the delivery of notifications is to drop all the routes, static and dynamic, from the project. Re-enabling then requires the recreation of the routes (easiest from an export file).

  This is a cumbersome process.

  Make this easier by adding a boolean config setting "suspended" (local, default no) which can be activated to temporarily disable delivery.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Currently all configuration keys accepts any value when getting set, regardless of sensibility. This is a hole in the defence against bad user input which should be closed ASAP. This requires some sort of mapping from key name to a validation type.

  This might be do-able with a parameterizable VT which can access the value of the parameter upon which to switch to the actual VT for the value.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The fx::mailgen package currently limits the display of field values to 2K characters (per field), to prevent the generation of overly large mails (*).

  Make this value configurable. The string uses to indicate the truncation is hardwired as well and could be made configurable as well.

  (*) Semi DOS attack by spamming a ticket system with a large description, whether serious (**) or not.
  (**) One of the bugs in the Tcl ticketing system has a 2 MB strace log as part of the bug description.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The internal "fx fossil get-manifest" proc retries the 'exec fossil artifact' in case of problems, limiting itself to a fixed number of 10 such.

  Make this number configurable.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The delay between retries is currently fixed to half a second (500 millis).
  Make this configurable as well.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Currently the repo location is shown as absolute path.
  Show it relative to CWD.
  Or show either absolute|relative, whichever is shorter.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The exact format of the generated mails is currently hardwired into fx.

  In the long run it will be better to have some sort of templating system with which users can customize the mails to suit their environment. The current format would become defaults.

  Note: This is an extension and/or generalization of [1e5e7b0c6d].


login: aku
mimetype: text/x-fossil-plain
icomment:
  Note that each mail requires 2 templates, for subject, and body.
  Possibly a third, to affect the mail headers as well.

  Other modes of transmission (IRC) will need their own templates too.


login: aku
mimetype: text/x-fossil-plain
icomment:
  The change notifications are currently fully geared to mailing these.
  On core.tcl.tk we have a separate system for sending notifications to IRC.

  Extend fx to be able to deliver IRC notifications as well.
  This requires an additional set of configuration information (how to inject IRC messages), plus an additional generator package as we cannot use mails for IRC.


login: aku
mimetype: text/x-fossil-plain
icomment:
  For some configuration keys (like "location") the information is per-repository, and having a global value makes no sense.

  Mark these keys in the code and block the use of --global for them.


login: aku
mimetype: text/x-fossil-plain
icomment:
  When generating the mail for a ticket change fx currently has hardwired knowledge about a number of possible fields (by name), namely the type of their values, and if we want them in the output (*).

  Given the configurability of fossil's ticket system this hardwired knowledge is bad in the long run.


  (*) The possible actions (so far) are:
   * SKIP - suppress, do not place in the mail
   * KEEP - default, show field in the mail
   * FORMAT - show field, is text, reformat to limit line-lengths
   * DATE - show field, is fractional julianday, format to ISO for readability.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Ticket [ef1ed37c8e] is an extension/generalization of this.

  Note also that the current actions intermingle field use and formatting.
  With a templating system at least use would be determined by the template.

  Of note, also, the current tabular format simply uses the field names as labels. We might wish to allow customization here as well.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Snarf code for progress animation in terminals from Tcllib (sak).
  Use it to judiciously add animations to various long-running commands.
  Make the animations switchable (i.e. have a quiet mode).


login: aku
mimetype: text/x-fossil-plain
icomment:
  See the cmdr ticket
    http://core.tcl.tk/akupries/cmdr/tktview/03940bc094
  Simply make use of that when it comes into existence.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Sending mails to addresses which are known to be bad is not nice and an unwarranted load on the mail infrastructure.

  Extend fx with commands to manage an exclusion list of mail addresses.
  Better even if the same facility allows us to map bad addresses into good ones.

  (The exclusion would then be a special case where the bad address maps into the empty string i.e. nothing, or a non-address string).


login: aku
mimetype: text/x-fossil-plain
icomment:
  The use of colorization in the terminal output is currently hardwired, especially in the mapping of symbolic names to colors.

  Make this configurable (new colors (*), changed mapping).
  While this is something which is more global than local, both should be supported I believe.


  (*) 256-color terminals.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Better, defer to Cmdr ticket
    http://core.tcl.tk/akupries/cmdr/tktview/8502a858bdd6
  and then switch over.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Modify cron_lock to use the "flock" command where available, for extended robustness. Keep the existing code for portability to platforms without "flock" (Windows, for example).


login: aku
mimetype: text/x-fossil-plain
icomment:
  Use TclX to catch the TERM signal where possible, to enable a clean exit in the face of a user abort.

  On Windows we will have to use TWAPI to complement TclX.

  Note: The app should keep working without the packages, in a degraded manner.


login: aku
mimetype: text/x-fossil-plain
icomment:
  Not getting spam, but spamming itself
  => Check number of pending changes and ignore if over a limit, plus mail to signal the issue.
  => Auto-suspend of notifications.

  => Need suspension flag to disable work without having to remove all routes.

  idea by A. Ferrieux


