Adding `Areas' to Req =================== John Sellens ;login: vol 22 no 1, February, 1997 Introduction ------------ Sooner or later every system administrator needs some kind of to-do list or problem tracking system. In Data Processing at the University of Waterloo, we've been using the "req" request tracking system[1] from Northeastern University since October, 1994. This article describes how we recently extended req to support "areas" -- a method of classifying requests into categories or areas of responsibility and providing more finely-grained access control. We've been very pleased with req -- it's a fairly straight-forward system for keeping track of problem reports and requests. It can be manipulated from the command line, via email, or through "tkreq", a GUI front end for the X Window System. (The most recent version of req is 1.2.7, from November, 1994.) There were, however, a couple of features that we were interested in seeing added to req. The first was some form of support for multiple queues -- the ability to divide the primary request queue up into separate queues for different areas of interest or responsibility (e.g. UNIX, networking, database, etc.). And the second was an alternative to the "all or nothing" permission model of req -- in order to read or manipulate requests (other than via email) a user had to be in the "request" group, which gave that user complete access to all the requests in the queue. This meant that in order to give one of our support people access to the request system, we had to trust them not to make any mistakes, and to use "correct" judgement (for our definition of "correct"), since they had complete access. Other req users have worked on extensions to req 1.2.7, most often to provide support for multiple queues, with the most notable likely being "multi-req" from the University of Utah. Multi-req was eventually abandoned in favour of the ongoing development of "RUST" [2] (version 1.0b5 of RUST was released in early December, 1996). At Waterloo, we have been hesitant to adopt a different tracking system, since req is working so well for us. It eventually occurred to us that certain additional functionality could be added to req fairly simply, which would likely provide the new features that we had been looking for. We hit on the idea of adding support for "areas" to req, as a way of classifying and dividing the single req request queue into multiple logical queues -- think of areas as "lightweight" queues. It turned out that it was easy to add support for areas to req, and our implementation provided access control to the multiple areas and, as a pleasant side effect, made it possible to provide read-only access to the requests. Implementation -------------- The req system is built around 3 primary programs - The "q" command displays the request queue, and provides a number of options to modify the selection of requests e.g. filter by owner, priority, status, etc. - The "req" command manipulates the queue e.g. resolves or assigns requests, merges two requests together, adds information to a request file, etc. - The "tkreq" command, which provides a GUI front end to the q and req commands. There are a number of other programs that are part of req, for such things as mail gatewaying, monitoring, and statistics reporting. Req is primarily written in Perl, with a small number of utility programs written in sh and C, and tkreq written in Tcl/Tk. Req keeps each request in a separate text file, with a number of keyword headers maintained in each file. It turned out that one of the headers that req already defined was an unused "X-Request-Areas" header. We decided to use that header (which was already created and maintained by req) to record the area to which the request was currently assigned. We decided to define the list of valid areas in req using a Perl associative array called "areanames". The names of the areas are the array's keys, and each entry in the array is a string containing an email address followed by a list of userids. The email address indicates where notifications concerning that area should be sent (and could, of course, be an alias), and the userids indicate who should be able to manipulate the requests in that area. In order to make it easy to modify the list of areas, the areanames array is defined in a separate "reqareas" file that is "require"d by the various Perl programs. The default area for requests is the empty string, which makes the default behaviour backwards compatible, and means that no changes are required to existing request files for use with our new version of req. It also, incidentally, makes the use of areas optional. Since we were already adding one external configuration file, we decided to add an additional "reqinit" file, which contains the information that defines and names the mailing list -- this information had previously been compiled into the programs. The use of these two external files caused some minor complications in tkreq, since it's not easy to include Perl code in a Tcl/Tk script. The next step was to modify the req command to "require" the two new files, and to add a "-area" flag to make it possible to assign a valid area to a given request. The modifications to the req command included a fairly simple-minded permissions check -- a user must be in the area's userid list in order to operate on (e.g. resolve, merge, take) a request currently assigned to that area. One effect of this policy is that it is possible to "give" a request to an area that you are not a member of, but it is not possible to "steal" a request from another area (unless you are also a member of that area). This permissions check also means that "daemon" (or whichever userid your mailer runs under) must be a member of each area, or there will be failures when trying to create, update or modify a request via email. (The sample reqareas file includes a "foreach" loop to add root and daemon to every area that has been defined.) The q command was then modified to add two options to allow selecting requests by area. The "-area" option allows the selection of requests based on the area they are currently assigned to, and the -noarea option provides an easy way to select requests that are not currently assigned to an area. Both of these options add an "Area" column to the output; otherwise the output format is unchanged. Leaving the default output format unchanged is very convenient, since the q command is often used by other programs which "know" its output format. Once the q and req commands had been modified, the tkreq command could be changed to take advantage of the new options. This turned out to be a somewhat more complicated task. The headers in tkreq suggest that it was written using version 2.2 of the "xf" interactive interface builder for Tcl/Tk. A little experimentation with modifying tkreq with this (relatively old) version of xf proved fruitless (I suspect I would have been more successful if I had bothered to read the documentation). So, I resorted to manually modifying the tkreq code directly (keeping Ousterhout's Tcl/Tk book [3] handy since I had no Tcl or Tk experience). tkreq was modified to make it possible to assign an area to a request (through a new "Areas" menu), and to allow filtering based on area (through a new entry in the "Filter" menu). Some information needed in tkreq was now available only through the Perl include files, reqareas and reqinit. After weighing the alternatives, we decided to create two new simple utility commands, req-areas, which produces a list of valid areas, and req-variables, which outputs a list of the variables that should have been defined in the reqinit file. This seemed to be easier (and conceptually less offensive) than parsing the Perl files with Tcl. tkreq is a 4,000 line, machine generated Tcl/Tk script, so it was sometimes a little challenging for a Tcl/Tk neophyte to find the appropriate place to make a change, but all things considered, it was fairly straight-forward. Conclusions ----------- So far, the addition of areas to req seems to be working well for us. It seems to provide a reasonable level of categorization and access control, and hasn't required us to make any substantial changes in the way we do things or the software that we use. One nice side effect of the changes has been that it is now possible to give someone read-only access to the requests. The requests are normally owned by a request userid and group, with read-write access for the owner, and read access for the group (the req command is setuid to the request userid, and works only for those in the request group). Since modifications to requests are now controlled by area membership, read-only access can be given to someone in the request group simply by not including them in any areas. (It should be noted that it is still possible to do anything using req's "X-request-do" email header, but an email message is sent to the request distribution list whenever an email message is sent to request, so the danger of unauthorized changes is lessened.) Once we had the idea of areas and how to implement them, the actual implementation was relatively straight-forward. An easy modification process usually means that either the modifications are well thought out, or that the original code was well designed and well written. In this case, the quality of the original code definitely made the modifications much easier to make. Availability ------------ Req with areas (labeled req-1.2a) is available from ftp://math.uwaterloo.ca/pub/req or through the author. References ---------- [1] Remy Evard, "Managing the Ever-Growing To-Do List", LISA 8 Proceedings, pp 111-116, San Diego, CA, 1994. [2] Craig Ruefenacht, "RUST: Managing Problem Reports and To-Do Lists", LISA 10 Proceedings, pp 81-89, Chicago, IL, 1996. [3] John K. Ousterhout, Tcl and the Tk Toolkit, Addison Wesley, 1994.