New Gadget Tutorial
Created by: Douglas Hunter
Permissions: read[all users] edit[all users] reply[all users]

Writing a new gadget

As mentioned before, a Gadgets installation is made up of one or more applications bundled as Par files. Sometimes we refer to an individual application as a gadget. A Gadgets installation typically employes several of these (commonly the Wiki or Blog for high-level functionality, Post for the actual document creation/edit interface, and Template for skinning the application).

Once we're up and running, to make a new gadget we change into our Gadgets checkout directory and run ./misc/admin/create_devel_gadget.pl.

And here is the output of create_devel_gadget.pl --help


  ./misc/admin/create_devel_gadget.pl --name "Gadget-Name"
     --implements "APIName" [ --implements "OtherAPIName" ]
     --uri_path "/uri/to/dispatch/"
     --comp_path "/path/to/mason/components/"
     [ --lib "/path/to/perl/lib/" --lib "/other/path/to/perl/lib/" ]
     [ --def "/path/to/comma/def" --def "/other/path/to/comma/def" ]

  OPTIONS:

    --name

      The name of the gadget you are creating.  This is required, and
      should be representative of the functionality the gadget
      provides.

    --implements

      The name(s) of the API(s) that this gadget implements.  If
      nothing is specified, it will default to the name of the gadget.
      Multiple APIs may be specified, if your gadget implements them.

    --uri_path

      The uri that will dispatch requests to this gadget.  This is
      required, and should be unique to this gadget.

    --comp_path

      The path to your development HTML::Mason components.  This is
      required, and the directory specified must be relative to the
      HTML::Mason component_root specified in your handler.pl (or
      otherwise specified in your Apache configuration).  For
      instance, if your component_root is /mycorp/webtree/, and your
      development components are in the directory
      /mycorp/webtree/kung-fu/, your would specify "/kung-fu/" as your
      comp_path.

    --lib

      The path(s) to a directory(s) containing any perl modules that will be
      bundled with this gadget.  This is typically used for bundling
      gadget specific logic that doesn't belong in the Gadgets
      standard library.

   --def

     The path(s) any XML::Comma document defition(s) that this gadget
     will use.  This is typically used for bundling a gadget specific
     doctype that doesn't belong in the Gadgets standard library.

Lets start by making a toy application called that lets folks write book reviews and then have discussions about those reviews.

We'll name this application "Jonas", after Jonas Wright. Jonas was flayed alive on August 4, 1632. His skin was then used to re-bind what was once his most favored posession, a book.

first, our project directory:

mkdir ~/prj/gadgets/Jonas

Then our "comma" directory. This is where Comma Document Definitions belong.

mkdir ~/prj/gadgets/Jonas/comma

"lib" for our Perl libraries:

mkdir ~/prj/gadgets/Jonas/lib

And finally "mason" for our HTML::Mason templates.

mkdir ~/prj/gadgets/Jonas/mason

And now we'll make a symlink of our project's HTML::Mason template directory into our component_root, at the path that I'll use later for my argument to --uri_path. Since I keep my component root down "/webtree":

ln -s ~/prj/gadgets/Jonas/mason /webtree/jonas

And make a stub for the Perl module used by Jonas by pasting these contents:


package Jonas;

use warnings;
use strict;

1;

to ~/prj/gadgets/Jonas/lib/Jonas.pm. This is where we'll refactor commonly used code from our Mason templates into.

Our application will need to store some meta-data about books being reviewed, so we'll also write a simple Comma Document Definition to define that metadata, and how it is stored by pasting these contents:

<DocumentDefinition>
<name>Review</name>

<element><name>post_key</name></element>

<element><name>title</name></element>
<element><name>ISBN</name></element>
<element><name>author</name></element>
<element><name>publisher</name></element>

<element>
  <name>rating</name>
  <macro>enum: 1 .. 5</macro>
</element>

<required> qw( post_key title ISBN author publisher rating ) </required>

<store>
  <name>main</name>
  <base>review</base>
  <location>GMT_3layer_dir</location>    
  <location>Sequential_file:'extension','.review'</location>
  <file_permissions>0664</file_permissions>
  <index_on_store>main</index_on_store>    
</store>

<index>
  <name>main</name>
  <field><name>post_key</name></field>
  <field><name>title</name></field>
  <field><name>ISBN</name></field>
  <field><name>author</name></field>
  <field><name>publisher</name></field>
  <field><name>rating</name></field>
</index>

</DocumentDefinition>

to ~/prj/gadgets/Jonas/defs/Review.def

Now that we have everything stubbed out, we run create_devel_gadget.pl with the following arguments:


./misc/admin/create_devel_gadget.pl --name Jonas \
                                      --uri_path "/review/" \
                                      --comp_path "/jonas/" \
                                      --lib /home/username/prj/gadgets/Jonas/lib/ \
                                      --def /home/dug/username/gadgets/Jonas/defs/Review.def

If everything has gone well so far, we should be able to echo "hello world!" >> /ob/webtree/jonas/test.html, load up http://localhost/review/ in a browser, and see our handiwork! (test.html can be deleted after verifying things are working).

We're going to keep this application very simple. There will be a form to create new reviews, and a page to list existing reviews.

Let's hack in these basic features. We'll start with our navigation and url dispatch. By creating /ob/webtree/jonas/autohandler that looks like this:

<%init>
  my $base = $g->interface_path( api => "Jonas" );
</%init>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>Jonas</title>
</head>

<div class="nav">
  <a href="<% $base . "new" %>">Write a Review</a> | <a href="<% $base . "list" %>">Read Reviews</a> </div>

% $m->call_next( %ARGS );

</html>

we add our html declaration and basic navigation to every page. Autohandler is a special component name in mason that says "execute me every request down this path".

Notice that <%init> block. We asked the gadgets API for the interface_path for "Jonas". That coresponds to the "--uri-path" argument we passed to create_devel_gadget.pl, and by using this path we make our application relocatable.

And now a basic index.html page:

<div class="welcome">
Welcome to Jonas, the book review gadget.  Why is this site called <a href="http://www.google.com/search?hl=en&q=%22Jonas+Wright%22+book+binding">Jonas</a>?
</div>

which automatically gets wrapped up by our autohandler (like all other components). This is the only component that isn't wrapped up by our URI dispatch component, the dhandler (show next). Calls to "http://localhost/review/" or "http://localhost/review/index.html" skip the dhandler (which only handles virtual requests) and directly call "index.html".

Next, we'll write a dhandler. Dhandlers in Mason handle virtual requests, and are a nice and easy way to do URI dispatching. We set up our URI dispatch by adding making a /ob/webtree/jonas/dhandler like so:

<%doc>
  /new     -> edit
  /edit/id -> edit?doc_id=id
  /read    -> list
  /read/id -> list?doc_id=id
</%doc>
%
<%once>
  use Jonas;
</%once>
%
<%perl>

  my $path = $m->dhandler_arg;

  if ( $path eq "new" ) {
    $m->comp( "edit" );
  } elsif ( $path =~ /^edit\/(.+)/ ) {
    my $id = $1;
    my $review = Jonas->get_review_by_id( $id );
    $review  or  $m->clear_and_abort( 404 );
    $m->comp( "edit", review_doc => $review );
  } elsif ( $path eq "read" ) {
    $m->comp( "list" );
  } elsif ( $path =~ /^read\/(.+)/ ) {
    my $id = $1;
    my $review = Jonas->get_review_by_id( $id );
    $review  or  $m->clear_and_abort( 404 );
    $m->comp( "list", review_doc => $review );
  } else {
    $m->clear_and_abort( 404 );
  }

</%perl>

We're using the as of yet unwritten edit and list components here to handle requests to /review/new, /review/read, etc.

Notice that we're also using the get_review_by_id method in our Jonas.pm library. Let's add that into /home/username/prj/Jonas/lib/Jonas.pm. It looks like so:


sub get_review_by_id {
  my ( $class, $id ) = @_;
  return  unless  $id;
  my $key = XML::Comma::Storage::Util->concat_key
              ( type => "Review", store => "main", id => $id );
  my $review = eval { XML::Comma::Doc->read( $key ) };
  return $review;
}

And now /ob/webtree/jonas/edit:

<%args>
  $title     => undef
  $ISBN      => undef
  $publisher => undef
  $author    => undef
  $rating    => undef
  $review    => undef
  %errors    => undef

  $review_doc => undef # for edits
</%args>
%
%
<%init>
  my $user = $g->auth->force_authn;

  my $post_doc;
  if ( $review_doc ) {
    $post_doc   = XML::Comma::Doc->read( $review_doc->post_key );
    $post_doc->check_authz( $user, 'write' )  or  $m->clear_and_abort( 403 );
  }

</%init>

<h3>New Book Review</h3>
<p>All fields are required.</p>
%if ( %errors ) {
  <div class="new-review-form-errors">
  <% join "<br />", values %errors %>
  </div>
% }
<form method="post" action="<% $g->interface_path( api => "Jonas" ) %>process">
% if ( $review_doc ) {
  <input type="hidden" name="review_key" value="<% $review_doc->doc_key %>" />
% }
  <table class="new-review-form">
    <tr>
      <td>Title:</td>
      <td>
        <input type="text" name="title" value="<% $title %>" />
      </td>
    </tr>
    <tr>
      <td>ISBN: </td>
      <td>
        <input type="text" name="ISBN" value="<% $ISBN %>" />
      </td>
    </tr>
    <tr>
      <td>Publisher: </td>
      <td>
        <input type="text" name="publisher" value="<% $publisher %>" />
      </td>
    </tr>
    <tr>
      <td>Author: </td>
      <td>
        <input type="text" name="author" value="<% $author %>" />
      </td>
    </tr>
    <tr>
      <td>Rating: </td>
      <td>
        <select name="rating">
% foreach my $n ( 1 .. 5 ) {
%   my $selected = ($rating and $rating == $n)  ?  'selected="1"'  : undef;
        <option value="<% $n %>" <% $selected %>><% $n %></option>
% }
        </select>
      </td>
    </tr>
    <tr>
      <td>Review: </td>
      <td><textarea rows="15" cols="40"
                    name="review"><% $review %></textarea>
    </tr>
    <tr>
      <td>Submit: </td>
      <td><input type="submit" name="submit" value="submit" /></td>
    </tr>
  </table>
</form>

This is just a form that posts to process. The most interesting line is probably $g->auth->force_authn, which uses our global $g object, which is our entry point to the Gadgets API. The auth object is documented at perldoc Gadgets::AuthManager, and tells us that force_authn forces authentication in order to access the resource it is called in. In other words, folks have to be logged in in order to write a review.

Another interesting couple of lines are:


  if ( $review_doc ) {
    $post_doc   = XML::Comma::Doc->read( $review_doc->post_key );
    $post_doc->check_authz( $user, 'write' )  or  $m->clear_and_abort( 403 );
  }

In a minute we'll see that if the POST to process is successful, it creates a Gadgets::Standard::Post doc that contains the review itself, along with a Review doc that contains a pointer to the Post, as well as the meta-data we need to collect to make a Post a Review.

If someone is trying to edit a review, we not only make sure that they are logged in, but that they are authorized to write to that post/review doc pair. By default the check_authz will return true for the owner of the doc, which is what we want.

And now for /ob/webtree/jonas/process:

<%args>
  $title     => undef
  $ISBN      => undef
  $publisher => undef
  $author    => undef
  $rating    => undef
  $review    => undef

  $review_key => undef
  %errors     => ()
</%args>

<%init>
  my $user = $g->auth->force_authn;

  my ( $review_doc, $post_doc );
  if ( $review_key ) {
    $review_doc = eval { XML::Comma::Doc->read( $review_key ) };
    $post_doc   = XML::Comma::Doc->read( $review_doc->post_key );
    $m->clear_and_abort( 404 )  if  $@;
    $post_doc->check_authz( $user, 'write' )  or  $m->clear_and_abort( 403 );
  }

  $title      or  $errors{ "title"     } = "Title is required.";
  $ISBN       or  $errors{ "ISBN"      } = "ISNB is required.";
  $publisher  or  $errors{ "publisher" } = "publisher is required.";
  $author     or  $errors{ "author"    } = "author is required.";
  $review     or  $errors{ "review" } = "review is required.";
  $rating =~ /^[1-5]$/  or  $errors{ "rating" } = "Rating must be 1 - 5.";

  if ( values %errors ) {
    $ARGS{ "review_doc" } = $review_doc  if  $review_doc;
    $m->comp( 'edit', %ARGS, errors => \%errors );
    return;
  }


  $review_doc ||= XML::Comma::Doc->new( type => "Review" );
  $post_doc   ||= XML::Comma::Doc->new( type => "Post"   );

  eval {
    # first make our post
    $post_doc->owner( $user->doc_key );
    $post_doc->app( "Jonas" );
    $post_doc->body->html( $review );
    $post_doc->store( store => "main" );

    # and then our review
    $review_doc->post_key( $post_doc->doc_key );
    $review_doc->title( $title );
    $review_doc->ISBN( $ISBN );
    $review_doc->publisher( $publisher );
    $review_doc->author( $author );
    $review_doc->rating( $rating );
    $review_doc->store( store => "main" );
  };
  if ( $@ ) {
    $errors{ "store_error" } = $@;
    $m->comp( "edit", %ARGS, errors => \%errors );
    return;
  } else {
    $m->redirect( "read/" . $review_doc->doc_id );
  }
</%init>

And for completenes, the list component:

<%args>
  $review_doc => undef
  $order_by   => "record_last_modified DESC"
</%args>

<%perl>

  if ( $review_doc ) {
    $m->comp( "listing", key => $review_doc->doc_key );
  } else {
    my $iterator = XML::Comma::Def->Review
                                  ->get_index( "main" )
                                  ->iterator( order_by => $order_by );
    while ( ++$iterator ) {
      $m->comp( "listing", key => $iterator->doc_key );
    }
  }

</%perl>


<%def listing>
<%args>
  $key
</%args>
<%init>
  my $review = XML::Comma::Doc->read( $key );
  my $post   = XML::Comma::Doc->read( $review->post_key );
</%init>

<% $post->body->html %>
<hr />
</%def>

And that's it, we now have a working Jonas application, that can be packaged up into a PAR file and deployed live!

To package it up: perl misc/admin/package_devel_gadget.pl --name Jonas --comp_root /ob/webtree/.

(and the output of package_devel_gadget.pl --help:


  misc/admin/package_devel_gadget.pl --name "Gadget-Name"
     --comp_path "/path/to/mason/components/"

  OPTIONS:

    --name GadgetName

      The name of the gadget you are packaging.  This is required, and
      must be a devel gadget (probably created with
      create_devel_gadget.pl --name).

    --comp_root /mason/comp_root/

      This is the HTML::Mason component_root specified in your
      handler.pl (or otherwise specified in your Apache
      configuration).  This is necissary for the script to find your
      Mason components (and is a required argument), which were
      specified relative to this path with the --comp_path option to
      create_devel_gadget.pl.

    --outfile /path/to/GadgetName.par

      Optionally tell package_devel_gadget.pl where to write out your
      new par file.  This defaults to a par file of the argument you
      provided --name (GadgetName.par) in your present working
      directory.

And install the production gadget: perl misc/admin/install_gadget.pl --par Jonas.par.

(and the output of install_gadget.pl --help:)


  misc/admin/install_gadget.pl --par "Gadget-Name.par"
     [--uri_path "/uri/to/dispatch/" ]
     [ --par_dir "/prod/pars/dir/" ]

  OPTIONS:

    --par

      The par that we're installing, probably generated by
      package_devel_gadget.pl.  This is required.

    --uri_path

      The uri that will dispatch requests to this gadget.  If
      unspecified the script will try to find a devel gadget of the
      same name and use its path.  It is requred when installing a
      production gadget on a system where the devel gadget doesn't
      exist.

    --pars_path

      The path to the directory where your production pars are
      installed.  This is typically somewhere down apache's
      "document_root", and needs to be readable by an apache child.
      The script will try to figure out your pars_path by looking at
      other non-devel installed gadgets, but will die if it can't
      figure out the path and its unspecified.  So it's kind of
      optional, at least after its been specified once.

The source to Jonas lives in https://chronicle.allafrica.com:8080/repository/branches/Gadgets-Oblong/misc/demo/