Introducing WebStencils

The latest version of RAD Studio, version 12.2, was recently released and among the many enhancements was the introduction of a new template engine called WebStencils. It's the new generation of text processing for WebBroker and RAD Server applications that produce HTML on web servers. But it's more than that as it can be used in Delphi or C++Builder applications of all kinds to generate JSON, XML, CSV, or any other template-based text output.

I was eager to learn this new method for Delphi-generated content as I had an immediate need to build a web server in Delphi. The best way I know how to learn a new technology is to compare it to something with which I already have experience. Since I've built WebBroker applications before, this was the natural place to start.

Building Two Simple WebBroker Programs

Therefore, my approach to learning WebStencils was to build two simple WebBroker apps, one using the old TPageProducer components and one using the new TWebStencilsProcessor components. These two simple apps were designed to do exactly the same thing and look nearly identical. This way, I was able to compare how similar tasks were done with each of the technologies.

The programs start with a login screen, present a list of customers, and take you to a customer edit screen by clicking on the ID of a customer in the list. The edit screen doesn't actually save anything and there are no complicated queries or fancy web frameworks. I just wanted to see how WebStencils works differently than PageProducers, so didn't want to take the time or muddy the code by writing a full-featured website for this learning exercise.

The two projects are both VCL WebApps, meaning they run as simple Windows apps that prompt for a port on which to listen, provide a button to start listening on that port for web requests, and another to open your default web browser to that port. It's what you get when you start a new Web Server Application in Delphi and select Stand-alone GUI application for the WebBroker Project Type.

The web module in each project has four TWebActionItems with the following PathInfo values:

• / (method: GET) default path prompts for a login
• /login (method: POST) authorizes the given username and password and if successful, redirects to /custlist
• /custlist (method: GET) lists some customers from a sample database
• /custedit (method: GET) presents the fields of a selected customer on an edit page

Both projects are on Github; the README there explains more details of how to set up the database and build the projects if you want to try them out.

Comparing PageProducers and WebStencils

As a quick review, the TPageProducer components take either strings or a file as a template and produce textual output (usually HTML). They do that by parsing each line and replacing special HTML-style tags. There are half a dozen built-in tags it already knows about like <#Link>, <#Image>, <#Table>, and a few more; but you'll likely create several of your own custom ones and handle them in OnHTMLTag events. A common way to generate a table of rows and columns is to embed the results of a TDataSetTableProducer into the content result of a PageProducer.

The TWebStencilsProcessor components work similarly: they take either strings or a file as a template and produce textual output (usually HTML). But that's where the similarities mostly end and the interesting differences start! Instead of looking for HTML tags and simply doing a textual replacement per tag, WebStencils uses an @ symbol that delineates either scripting keywords, objects registered with the WebStencils engine, or (like the PageProducers) return simple textual replacements by using OnGetValue events. The scripting keywords can contain conditional expressions, import a file, or iterate over a data set to generate a table of rows and columns (without needing to use an additional component and providing much more flexibility).

The concepts and keywords I'll cover in this article include:

• @Layout
• @RenderBody
• @Import
• @object.property
• @LoginRequired
• @foreach { }
• @if { }

I'll start by looking at the first two and explain the framework used in these demos.

Laying Out the Framework

I tried to keep all the HTML in separate files for easier management and comparison. The exception was specifying a common web framework (header and footer) using PageProducers. For this, I used embedded strings in two additional TPageProducer components (ppPageHeader and ppPageFooter) to contain the HTML parts for each. Therefore, each PageProducer's HTML content would look like this:

<#Header>
  <h2>Content Title</h2>
  ... content ...
<#Footer>

and the PageProducer's OnHTMLTag event would have to replace the <#header>, and <#footer>, custom tags with the top and bottom part of the HTML page:

procedure TwebCustListWebBroker.ppAllHTMLTags(...);
begin
  if SameText(TagString, 'Header') then
    ReplaceText := ppPageHeader.Content
  else if SameText(TagString, 'Footer') then
    ReplaceText := ppPageFooter.Content;
end; 

Using WebStencils, I was able to fully separate out a framework file that can wrap around the content body (resulting in one more HTML file but two less components on the web module for the that project). This is accomplished by using the @LayoutPage keyword like this:

@LayoutPage FrameworkFilename.html
  <h2>Content Title</h2>
  ... content ...

In this case, the entire framework for the website (header and footer) is contained in one file making it much easier to work with in an HTML editor because you don't have to manually paste the strings from the header and footer text in the two PageProducers. The only thing you need to do to make it a WebStencils framework page is place the WebStencils keyword, @RenderBody in where each page's content will go. Then, each generated page that references this framework, automatically gets it's content inserted into this framework file producing the resulting HTML page.

Here's a stripped down version of the framework page I built for this demo:

<!DOCTYPE html>
<html lang="en">
<body>
  <div class="container px-5">
    @RenderBody
  <div><em>Customer List Demo</em></div>
</body>
</html>

It's easy to switch the framework by simply giving the @LayoutPage a different filename. And an extra bonus for using WebStencils is there's no extra Delphi code needed to implement this.

Importing External Files

Web page headers can get very large, especially if styles are defined there. For this reason, styles are often listed in .CSS files and referenced in <script> tags of the HTML header. I'm going to add styles in a separate file as well but will let WebBroker include them directly into the generated HTML. Let's see how to include external files using these two text processing paradigms.

There's nothing special to importing an external file using PageProducers, simply create another custom tag and add another conditional statement in the OnHTMLTag event to use the content from another PageProducer.

Here's the HTML in the header text:

<style>
  <#IncludeCSS>
</style> 

And the Delphi code in the OnHTMLTag event is straight-forward, where ppStyles is a TPageProducer linked to a file named, "style.css":

procedure TwebCustListWebBroker.ppAllHTMLTags(...; var ReplaceText: string); 
begin
  if SameText(TagString, 'IncludeCSS') then
    ReplaceText := ppStyles.Content;
end;

The WebStencils version, however, provides a magic keyord that, again, saves us a little code.

Here's the HTML:

<style>
  @Import style.css
</style> 

No need for any Delphi code here, WebStencils knows how to include an external file and the parameter to the keyword specifies the filename.

Replacing Template Variables

At the top of each web page in these demos is listed the application name and the version. For the PageProducer versions, custom HTML tags are used and the HTML snippet for them looks like this:

<h2><#AppName></h2>
<h4>Version <#AppVersion></h4>

Here, the <#AppName> tag is replaced in the PageProducer's OnHTMLTag event with a constant defined in the Delphi app for the application name; same with the <#AppVersion> tag.

procedure TwebCustListWebBroker.ppHTMLTags(...);
begin
  if SameText(TagString, 'AppName') then
    ReplaceText := APP_NAME
  else if SameText(TagString, 'AppVersion') then
    ReplaceText := APP_VERSION;
end;

In the WebStencils version, the framework file has something very similar:

<h2>@App.Title</h2>
<h4>Version @App.Version</h4>

Instead of HTML tags, you'll see an @ symbol with an object name and dotted property name that gets replaced. I could have used an OnGetValue event to mimic the technique used by the PageProducers; instead, I registered an object (the form) with the TWebStencilsEngine component on the web module and named it App, with FTitle and FVersion as private fields on the form but surfaced as public properties (without the "F") so that they're accessible by the WebStencilsEngine:

procedure TwebCustListWebStencil.WebModuleCreate(Sender: TObject);
begin
  FTitle := 'Customer List for WebStencils';
  FVersion := '0.3';
  wsEngineCustList.AddVar('App', Self, False);
end;

Now, any time a WebStencils processor component tied to that engine encounters an object in the text named @App, it knows to look at that registered object and replace the text with the value of that named property--no need for an event handler!

Requiring Authorization

Before I show the customer list, I need to make sure the user is authorized to view it. In the PageProducers version, I created a Boolean property, FIsLoggedIn and initialized it to False. Then, the /login action sets it to True and redirects to the /custlist page if authenticated or displays the error message shown above if not:

procedure TwebCustListWebBroker.LoginVerifyAction(...);
var
  Username, Password: string;
begin
  Username := Request.ContentFields.Values['uname'];
  Password := Request.ContentFields.Values['psw'];
  if dmCust.LoginCheck(Username, Password) then
  begin
    FIsLoggedIn := True;
    Response.SendRedirect('/custlist');
  end else
    Response.Content := ppLoginFailed.Content;
  Handled := True;
end;

Each page needs to check to see if the user is logged in. To do this with PageProducers, each web action needs to check FIsLoggedIn and either returns the requested content if authorized or shows the "Access Denied" page:

procedure TwebCustListWebBroker.ListCustomersAction(...);
begin
  if IsLoggedIn then
    Response.Content := ppCustList.Content
  else
    Response.Content := ppAccessDenied.Content;
  Handled := True;
end;

Authentication is a common need for accessing information these days. Conveniently, WebStencils processors provide a way to prevent their content from showing unless the user is logged in. This is accomplished by including the following keyword at the top of the HTML:

@LoginRequired

In order to tell the processor that the user is authenticated and to go ahead and show the content, we need to set a property for each processor that might need to know:

procedure TwebCustListWebStencils.LoginVerifyAction(...);
var
  Username, Password: string;
begin
  Username := Request.ContentFields.Values['uname'];
  Password := Request.ContentFields.Values['psw'];
  if dmCust.LoginCheck(Username, Password) then
  begin
    wspCustList.UserLoggedIn := True;
    wspCustEdit.UserLoggedIn := True;
    Response.SendRedirect('/custlist');
  end else
    Response.Content := ppLoginFailed.Content;
  Handled := True;
end;

There is no other Delphi code required once those properties are set but if the user tries to access a page with the keyword listed and the processor does not think the user is logged in, the WebStencils engine will throw an exception that can be caught in the processor for the page on which authentication failed:

...
try
  Response.Content := wspCustList.Content;
except
  on E:EWebStencilsLoginRequired do
    Response.Content := wspAccessDenied.Content;
end;
...

If you don't handle it yourself, an Internal Application Error will be displayed to the user:

Iterating Over a Dataset

Now that I've got a framework in place with included styles and the user has been properly authenticated, it's time to show some data!

The sample database for this application has a few customers that I'll list in an HTML table. I used an HTML table instead of a CSS table to be compatible with the PageProducers version of these projects.

I showed the code for the /custlist action handler above (see the "ListCustomersAction" method); it simply returns the content of the ppCustList PageProducer, which is this HTML:

<#header>
  <h2>Customers</h2>
  <#customers>
<#footer>

The OnHTMLTag event returns the content of the TDataSetTableProducer in place of the <#customers> tag:

procedure TwebCustListWebBroker.ppAllHTMLTags(...);
begin
  if SameText(TagString, 'Customers') then
    ReplaceText := pptblCustomers.Content;
end;

The TDataSetTableProducer is convenient in that, given a dataset, it will open and close it for you, wrap each row of data in a <tr> tag and each field of data in a <td> tag, and allow some customization with its properties and a couple of event handlers for styling and custom processing (which I'll get to in a bit). But it is stuck in the early days of writing HTML.

This is the part of WebStencils that begins to show the powerful scripting capabilities of this engine. Instead of tying our dataset to a component that only knows how to do one thing (build an HTML table), I'll use the @foreach keyword in WebStencils to iterate over the records of a dataset. I'll register the dataset with the WebStencilsEngine, open it before and close it after the content is generated but leave the format of each field and row to be completely defined in the HTML template.

First, here's the WebActionHandler for /custlist (minus the exception block for user authentication I showed earlier):

procedure TwebCustListWebStencil.ListCustomersAction(...);
begin
  dmCust.qryCustomers.Open;
  try
    if not wsEngineCustList.HasVar('CustList') then
      wsEngineCustList.AddVar('CustList', dmCust.qryCustomers, False);
    Response.Content := wspCustList.Content;
  finally
    dmCust.qryCustomers.Close;
  end;
end;

First, I check to see if the object name we're giving for the dataset, CustList, is already registered by using the HasVar function; if not, I add it. Then I simply return the content of the WebSencils processor, which is the HTML template for the customer list:

@LayoutPage CustListFramework1.html
@LoginRequired

<h2>Customers</h2>
<table CellSpacing=10 CellPadding=8 Border=1>
  <tr>
    <th Align="right">ID</th>
    <th Align="left">First</th>
    <th Align="left">Last</th>
    <th Align="left">Company</th>
  </tr>
  @foreach CustList {
    <tr>
      <td align="right" ><a href="\[email protected]">@loop.CustomerId</a></td>
      <td>@loop.FirstName</td>
      <td>@loop.LastName</td>
      <td>@loop.Company</td>
    </tr>
  }
</table>

The @foreach keyword iterates over each row of the registered dataset (CustList). The template for each table row is given within the curly braces; in our demo case, it starts and ends with a <tr> tag. The WebStencils syntax provides the @loop keyword which acts as a temporary object for each row, providing acces to the fields CustomerID, FirstName, LastName, and Company.

Again, I used HTML tables only so the resulting web pages produced by these two apps would be the same. This list could just as easily have been generated with CSS or a JavaScript library.

Including Template Text Conditionally

There is a lot more to WebStencils but I'm only going to touch on one more keyword. By reviewing the output of the generated customer list, I noticed not all customers have a company name, so I made a small modification to the output to color the rows that have a company name to make them stand out.

To do that, I added a calculated field to the dataset called IsBusiness that I can check while generating each row; then I added a style class in the HTML header called company_row to add to the <td> tags on rows that have a non-blank company name.

With PageProducers, this is all done in Delphi code using the OnFormatCell event handler of the TDatsSetTableProducer where I set the CustomAttrs property of all fields (except CustomerID) based on whether the IsBusiness calculated field is True or not:

procedure TwebCustListWebBroker.pptblCustomersFormatCell(...);begin
  if CellRow > 0 then begin
    if CellColumn = 0 then // CustomerID
      CellData := Format('<a href="\custedit?cust_no=%s">%s</a>', [CellData, CellData])
    else if pptblCustomers.DataSet.FieldByName('IsBusiness').AsBoolean then
      CustomAttrs := 'class="company_row"';
  end;
end;

In the WebStencils version, the only thing that changes is the HTML template because I can use the @if keyword to provide conditional processing right within the template! The @foreach loop that generates each dataset row now looks like this:

@foreach CustList {
  <tr>
    <td align="right" ><a href="\[email protected]">@loop.CustomerId</a></td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.FirstName</td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.LastName</td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.Company</td>
  </tr>
}

Once again, curly braces are used to delineate the block of text that should be generated if the expression loop.IsBusiness evaluates to True for those rows.

Going Further

I only touched on a few things; there are more keywords and properties and events you can use. I also didn't explain the difference between the TWebStencilsEngine and the TWebStencilsProcessor components; I mainly wanted to show how everything worked; read the WebStencils documentation for details.

Also, be sure and check out the recent Embarcadero blog, RAD Studio Web Development Reimagined with the WebStencils Template Engine. It mentions there are samples and possibly a book coming to elaborate on this powerful new text-processing engine.