Coding in Delphi and Beyond - Thoughts from David Cornelius

AI Saves the Day

david — Sun, 27 Oct 2024 23:20:05 +0000

AI Saves the Day

Category

Miscellaneous

david Sun, 10/27/2024 - 16:20

An online customer encountered a rare problem with a database which stumped me; AI came to the rescue.

A friend of mine sells an insurance management system to agencies around the U.S. It's been a successful niche product for over 20 years, is written in Delphi 2007 so runs only on Windows, and uses the Firebird database on the back end. Until a few years ago, his customers all hosted their own local database server (or a designated user workstation) to hold the data. The application is very solid and chugs along with virtually no support calls related to corrupt databases.

About six years ago, his customers were clamoring for an online solution so he asked me to help and we set up a couple of servers and started hosting his software and database in the cloud. One of the servers runs the Firebird database which easily handles the 70-90 users among a couple of dozen insurance offices.

Recently we ran into a rare database problem. Every once in a great while, a PC will get disconnected from the database. Most of the time, the server recognizes it and cleans up the connection but in rare cases, it thinks the user is still connected. When this happens, the users is blocked and the only recourse in the past has been to restart the office database server. Each insurance agency has their own database but in the cloud, all the database requests go through one database server, so I didn't want to disrupt the other 80 or so users to get this one user back online.

I knew there must be a fix but I don't use the Firebird database very often and encounter errors like this even less so wasn't sure how to rectify the situation. After mulling over several possible ways to handle this, each with multiple downsides, I decided to ask AI for a suggestion.

Here's the short conversation I had with Claude:

DC: how do you tell firebird 2.15 to clear connections from a database?

AI: To clear connections from a Firebird 2.15 database, you can use the gfix utility with the -shut command. Here's how you can do it:

1. Open a command prompt or terminal.
2. Use the gfix command with the following syntax:
gfix -user SYSDBA -password masterkey -shut -force 0 database_path
Replace masterkey with your SYSDBA password if it's different, and database_path with the full path to your Firebird database file.

The -shut option shuts down the database, and -force 0 disconnects all users immediately.

3. After running this command, the database will be shut down and all connections will be cleared.
4. To bring the database back online, use:
gfix -user SYSDBA -password masterkey -online database_path
This method forcibly disconnects all users, so use it cautiously in a production environment. If you need a less disruptive approach or more specific control over which connections to clear, you might need to use Firebird's monitoring tables to identify specific connections and then use the API to terminate them individually.

Would you like me to explain any part of this process in more detail?

I was able to apply this fix quickly and it only affected this one insurance office, which was soon back to work.

This is the main reason I'm loving AI and using it more and more: it can find things I've forgotten about very quickly. It also allows me to be a jack of many trades and look like a master of all!

Only problem with AI …

Thomas Müller (not verified) Mon, 10/28/2024 - 00:49

Only problem with AI (including Claude) is that it provides plausible sounding answers that are wrong or misleading. A few days ago I asked Claude for a Delphi implementation of Timsort. What I got looked plausible but it simply didn't work. It sorted about 10 entries correctly and but broke for more. The same happened with ChatGPT on other occasions with completely other topics. That makes me wary of anything those hyped "helpers" suggest.

Being wary is good.

david Mon, 10/28/2024 - 08:36

I do not promote the idea that one should take a suggested AI answer and use it as a solution without verification. What I have found is that it either jump-starts ideas and thinking about how to solve a problem or, as in the case mentioned in this blog, reminds me of a tool I can use. I checked the documentation, verified the syntax, ran the command, and verified with the user the problem was fixed.

AI is not doing my work for me, it's not going to replace my job as a professional programmer that understands user needs or application context or can test to see if a suggestion is a proper solution. It is, though, becoming a helpful research assistant.

Add new comment

Introducing WebStencils

david — Thu, 26 Sep 2024 05:22:10 +0000

Introducing WebStencils

Category

Programming

david Wed, 09/25/2024 - 22:22

The latest version of RAD Studio, version 12.2, was recently released and among the many enhancements was the introduction of a new template engine called WebStencils. It's the new generation of text processing for WebBroker and RAD Server applications that produce HTML on web servers. But it's more than that as it can be used in Delphi or C++Builder applications of all kinds to generate JSON, XML, CSV, or any other template-based text output.

I was eager to learn this new method for Delphi-generated content as I had an immediate need to build a web server in Delphi. The best way I know how to learn a new technology is to compare it to something with which I already have experience. Since I've built WebBroker applications before, this was the natural place to start.

Building Two Simple WebBroker Programs

Therefore, my approach to learning WebStencils was to build two simple WebBroker apps, one using the old TPageProducer components and one using the new TWebStencilsProcessor components. These two simple apps were designed to do exactly the same thing and look nearly identical. This way, I was able to compare how similar tasks were done with each of the technologies.

The programs start with a login screen, present a list of customers, and take you to a customer edit screen by clicking on the ID of a customer in the list. The edit screen doesn't actually save anything and there are no complicated queries or fancy web frameworks. I just wanted to see how WebStencils works differently than PageProducers, so didn't want to take the time or muddy the code by writing a full-featured website for this learning exercise.

The two projects are both VCL WebApps, meaning they run as simple Windows apps that prompt for a port on which to listen, provide a button to start listening on that port for web requests, and another to open your default web browser to that port. It's what you get when you start a new Web Server Application in Delphi and select Stand-alone GUI application for the WebBroker Project Type.

The web module in each project has four TWebActionItems with the following PathInfo values:

/ (method: GET) default path prompts for a login
/login (method: POST) authorizes the given username and password and if successful, redirects to /custlist
/custlist (method: GET) lists some customers from a sample database
/custedit (method: GET) presents the fields of a selected customer on an edit page

Both projects are on Github; the README there explains more details of how to set up the database and build the projects if you want to try them out.

Comparing PageProducers and WebStencils

As a quick review, the TPageProducer components take either strings or a file as a template and produce textual output (usually HTML). They do that by parsing each line and replacing special HTML-style tags. There are half a dozen built-in tags it already knows about like <#Link>, <#Image>, <#Table>, and a few more; but you'll likely create several of your own custom ones and handle them in OnHTMLTag events. A common way to generate a table of rows and columns is to embed the results of a TDataSetTableProducer into the content result of a PageProducer.

The TWebStencilsProcessor components work similarly: they take either strings or a file as a template and produce textual output (usually HTML). But that's where the similarities mostly end and the interesting differences start! Instead of looking for HTML tags and simply doing a textual replacement per tag, WebStencils uses an @ symbol that delineates either scripting keywords, objects registered with the WebStencils engine, or (like the PageProducers) return simple textual replacements by using OnGetValue events. The scripting keywords can contain conditional expressions, import a file, or iterate over a data set to generate a table of rows and columns (without needing to use an additional component and providing much more flexibility).

The concepts and keywords I'll cover in this article include:

@Layout
@RenderBody
@Import
@object.property
@LoginRequired
@foreach { }
@if { }

I'll start by looking at the first two and explain the framework used in these demos.

Laying Out the Framework

I tried to keep all the HTML in separate files for easier management and comparison. The exception was specifying a common web framework (header and footer) using PageProducers. For this, I used embedded strings in two additional TPageProducer components (ppPageHeader and ppPageFooter) to contain the HTML parts for each. Therefore, each PageProducer's HTML content would look like this:

<#Header>
  <h2>Content Title</h2>
  ... content ...
<#Footer>

and the PageProducer's OnHTMLTag event would have to replace the <#header>, and <#footer>, custom tags with the top and bottom part of the HTML page:

procedure TwebCustListWebBroker.ppAllHTMLTags(...);
begin
  if SameText(TagString, 'Header') then
    ReplaceText := ppPageHeader.Content
  else if SameText(TagString, 'Footer') then
    ReplaceText := ppPageFooter.Content;
end;

Using WebStencils, I was able to fully separate out a framework file that can wrap around the content body (resulting in one more HTML file but two less components on the web module for the that project). This is accomplished by using the @LayoutPage keyword like this:

@LayoutPage FrameworkFilename.html
  <h2>Content Title</h2>
  ... content ...

In this case, the entire framework for the website (header and footer) is contained in one file making it much easier to work with in an HTML editor because you don't have to manually paste the strings from the header and footer text in the two PageProducers. The only thing you need to do to make it a WebStencils framework page is place the WebStencils keyword, @RenderBody in where each page's content will go. Then, each generated page that references this framework, automatically gets it's content inserted into this framework file producing the resulting HTML page.

Here's a stripped down version of the framework page I built for this demo:

<!DOCTYPE html>
<html lang="en">
<body>
  <div class="container px-5">
    @RenderBody
  <div><em>Customer List Demo</em></div>
</body>
</html>

It's easy to switch the framework by simply giving the @LayoutPage a different filename. And an extra bonus for using WebStencils is there's no extra Delphi code needed to implement this.

Importing External Files

Web page headers can get very large, especially if styles are defined there. For this reason, styles are often listed in .CSS files and referenced in <script> tags of the HTML header. I'm going to add styles in a separate file as well but will let WebBroker include them directly into the generated HTML. Let's see how to include external files using these two text processing paradigms.

There's nothing special to importing an external file using PageProducers, simply create another custom tag and add another conditional statement in the OnHTMLTag event to use the content from another PageProducer.

Here's the HTML in the header text:

<style>
  <#IncludeCSS>
</style>

And the Delphi code in the OnHTMLTag event is straight-forward, where ppStyles is a TPageProducer linked to a file named, "style.css":

procedure TwebCustListWebBroker.ppAllHTMLTags(...; var ReplaceText: string); 
begin
  if SameText(TagString, 'IncludeCSS') then
    ReplaceText := ppStyles.Content;
end;

The WebStencils version, however, provides a magic keyord that, again, saves us a little code.

Here's the HTML:

<style>
  @Import style.css
</style>

No need for any Delphi code here, WebStencils knows how to include an external file and the parameter to the keyword specifies the filename.

Replacing Template Variables

At the top of each web page in these demos is listed the application name and the version. For the PageProducer versions, custom HTML tags are used and the HTML snippet for them looks like this:

<h2><#AppName></h2>
<h4>Version <#AppVersion></h4>

Here, the <#AppName> tag is replaced in the PageProducer's OnHTMLTag event with a constant defined in the Delphi app for the application name; same with the <#AppVersion> tag.

procedure TwebCustListWebBroker.ppHTMLTags(...);
begin
  if SameText(TagString, 'AppName') then
    ReplaceText := APP_NAME
  else if SameText(TagString, 'AppVersion') then
    ReplaceText := APP_VERSION;
end;

In the WebStencils version, the framework file has something very similar:

<h2>@App.Title</h2>
<h4>Version @App.Version</h4>

Instead of HTML tags, you'll see an @ symbol with an object name and dotted property name that gets replaced. I could have used an OnGetValue event to mimic the technique used by the PageProducers; instead, I registered an object (the form) with the TWebStencilsEngine component on the web module and named it App, with FTitle and FVersion as private fields on the form but surfaced as public properties (without the "F") so that they're accessible by the WebStencilsEngine:

procedure TwebCustListWebStencil.WebModuleCreate(Sender: TObject);
begin
  FTitle := 'Customer List for WebStencils';
  FVersion := '0.3';
  wsEngineCustList.AddVar('App', Self, False);
end;

Now, any time a WebStencils processor component tied to that engine encounters an object in the text named @App, it knows to look at that registered object and replace the text with the value of that named property--no need for an event handler!

Requiring Authorization

Before I show the customer list, I need to make sure the user is authorized to view it. In the PageProducers version, I created a Boolean property, FIsLoggedIn and initialized it to False. Then, the /login action sets it to True and redirects to the /custlist page if authenticated or displays the error message shown above if not:

procedure TwebCustListWebBroker.LoginVerifyAction(...);
var
  Username, Password: string;
begin
  Username := Request.ContentFields.Values['uname'];
  Password := Request.ContentFields.Values['psw'];
  if dmCust.LoginCheck(Username, Password) then
  begin
    FIsLoggedIn := True;
    Response.SendRedirect('/custlist');
  end else
    Response.Content := ppLoginFailed.Content;
  Handled := True;
end;

Each page needs to check to see if the user is logged in. To do this with PageProducers, each web action needs to check FIsLoggedIn and either returns the requested content if authorized or shows the "Access Denied" page:

procedure TwebCustListWebBroker.ListCustomersAction(...);
begin
  if IsLoggedIn then
    Response.Content := ppCustList.Content
  else
    Response.Content := ppAccessDenied.Content;
  Handled := True;
end;

Authentication is a common need for accessing information these days. Conveniently, WebStencils processors provide a way to prevent their content from showing unless the user is logged in. This is accomplished by including the following keyword at the top of the HTML:

@LoginRequired

In order to tell the processor that the user is authenticated and to go ahead and show the content, we need to set a property for each processor that might need to know:

procedure TwebCustListWebStencils.LoginVerifyAction(...);
var
  Username, Password: string;
begin
  Username := Request.ContentFields.Values['uname'];
  Password := Request.ContentFields.Values['psw'];
  if dmCust.LoginCheck(Username, Password) then
  begin
    wspCustList.UserLoggedIn := True;
    wspCustEdit.UserLoggedIn := True;
    Response.SendRedirect('/custlist');
  end else
    Response.Content := ppLoginFailed.Content;
  Handled := True;
end;

There is no other Delphi code required once those properties are set but if the user tries to access a page with the keyword listed and the processor does not think the user is logged in, the WebStencils engine will throw an exception that can be caught in the processor for the page on which authentication failed:

...
try
  Response.Content := wspCustList.Content;
except
  on E:EWebStencilsLoginRequired do
    Response.Content := wspAccessDenied.Content;
end;
...

If you don't handle it yourself, an Internal Application Error will be displayed to the user:

Iterating Over a Dataset

Now that I've got a framework in place with included styles and the user has been properly authenticated, it's time to show some data!

The sample database for this application has a few customers that I'll list in an HTML table. I used an HTML table instead of a CSS table to be compatible with the PageProducers version of these projects.

I showed the code for the /custlist action handler above (see the "ListCustomersAction" method); it simply returns the content of the ppCustList PageProducer, which is this HTML:

<#header>
  <h2>Customers</h2>
  <#customers>
<#footer>

The OnHTMLTag event returns the content of the TDataSetTableProducer in place of the <#customers> tag:

procedure TwebCustListWebBroker.ppAllHTMLTags(...);
begin
  if SameText(TagString, 'Customers') then
    ReplaceText := pptblCustomers.Content;
end;

The TDataSetTableProducer is convenient in that, given a dataset, it will open and close it for you, wrap each row of data in a <tr> tag and each field of data in a <td> tag, and allow some customization with its properties and a couple of event handlers for styling and custom processing (which I'll get to in a bit). But it is stuck in the early days of writing HTML.

This is the part of WebStencils that begins to show the powerful scripting capabilities of this engine. Instead of tying our dataset to a component that only knows how to do one thing (build an HTML table), I'll use the @foreach keyword in WebStencils to iterate over the records of a dataset. I'll register the dataset with the WebStencilsEngine, open it before and close it after the content is generated but leave the format of each field and row to be completely defined in the HTML template.

First, here's the WebActionHandler for /custlist (minus the exception block for user authentication I showed earlier):

procedure TwebCustListWebStencil.ListCustomersAction(...);
begin
  dmCust.qryCustomers.Open;
  try
    if not wsEngineCustList.HasVar('CustList') then
      wsEngineCustList.AddVar('CustList', dmCust.qryCustomers, False);
    Response.Content := wspCustList.Content;
  finally
    dmCust.qryCustomers.Close;
  end;
end;

First, I check to see if the object name we're giving for the dataset, CustList, is already registered by using the HasVar function; if not, I add it. Then I simply return the content of the WebSencils processor, which is the HTML template for the customer list:

@LayoutPage CustListFramework1.html
@LoginRequired

<h2>Customers</h2>
<table CellSpacing=10 CellPadding=8 Border=1>
  <tr>
    <th Align="right">ID</th>
    <th Align="left">First</th>
    <th Align="left">Last</th>
    <th Align="left">Company</th>
  </tr>
  @foreach CustList {
    <tr>
      <td align="right" ><a href="\[email protected]">@loop.CustomerId</a></td>
      <td>@loop.FirstName</td>
      <td>@loop.LastName</td>
      <td>@loop.Company</td>
    </tr>
  }
</table>

The @foreach keyword iterates over each row of the registered dataset (CustList). The template for each table row is given within the curly braces; in our demo case, it starts and ends with a <tr> tag. The WebStencils syntax provides the @loop keyword which acts as a temporary object for each row, providing acces to the fields CustomerID, FirstName, LastName, and Company.

Again, I used HTML tables only so the resulting web pages produced by these two apps would be the same. This list could just as easily have been generated with CSS or a JavaScript library.

Including Template Text Conditionally

There is a lot more to WebStencils but I'm only going to touch on one more keyword. By reviewing the output of the generated customer list, I noticed not all customers have a company name, so I made a small modification to the output to color the rows that have a company name to make them stand out.

To do that, I added a calculated field to the dataset called IsBusiness that I can check while generating each row; then I added a style class in the HTML header called company_row to add to the <td> tags on rows that have a non-blank company name.

With PageProducers, this is all done in Delphi code using the OnFormatCell event handler of the TDatsSetTableProducer where I set the CustomAttrs property of all fields (except CustomerID) based on whether the IsBusiness calculated field is True or not:

procedure TwebCustListWebBroker.pptblCustomersFormatCell(...);begin
  if CellRow > 0 then begin
    if CellColumn = 0 then // CustomerID
      CellData := Format('<a href="\custedit?cust_no=%s">%s</a>', [CellData, CellData])
    else if pptblCustomers.DataSet.FieldByName('IsBusiness').AsBoolean then
      CustomAttrs := 'class="company_row"';
  end;
end;

In the WebStencils version, the only thing that changes is the HTML template because I can use the @if keyword to provide conditional processing right within the template! The @foreach loop that generates each dataset row now looks like this:

@foreach CustList {
  <tr>
    <td align="right" ><a href="\[email protected]">@loop.CustomerId</a></td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.FirstName</td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.LastName</td>
    <td @if loop.IsBusiness { class="company_row" }>@loop.Company</td>
  </tr>
}

Once again, curly braces are used to delineate the block of text that should be generated if the expression loop.IsBusiness evaluates to True for those rows.

Going Further

I only touched on a few things; there are more keywords and properties and events you can use. I also didn't explain the difference between the TWebStencilsEngine and the TWebStencilsProcessor components; I mainly wanted to show how everything worked; read the WebStencils documentation for details.

Also, be sure and check out the recent Embarcadero blog, RAD Studio Web Development Reimagined with the WebStencils Template Engine. It mentions there are samples and possibly a book coming to elaborate on this powerful new text-processing engine.

Example failed (for me)

Joe Donth (not verified) Sat, 11/02/2024 - 09:13

I copied your examples, copies the DB and when I try to run the WebBroker version I get:

Project CustListWebBroker.exe raised exception class EFDException with message '[FireDAC][Stan][Def]-254. Definition [WebStencilsChinook] is not found in [C:\Users\Public\Documents\Embarcadero\Studio\FireDAC\FDConnectionDefs.ini]'.

Thoughts?

Two ways to set up the database

david Sat, 11/02/2024 - 14:56

There are two ways to specify where the database file resides, 1) by creating a FDConnection Definition or 2) simply specifying all the database parameters directly in the FDConnection component itself.

I chose the first option: Expand the FireDAC Providers in the Data Explorer pane in Delphi, and create a new connection definition for SQLite, I pointed it to the database file, and gave it a name. Then, in the FDConnection Editor of the TFDConnection component on the project's data module, I selected that saved definition in the "Connection Definition Name" drop-down list.

What you can do instead is set the properties of the data module's TFDConnection component directly: select the SQLite Driver ID (leaving the Connection Definition Name box blank), point it to your database file, and optionally set any other parameters you might need.

Either way, you have to specify the Driver ID and database filename, either in a saved connection in the Data Explorer, or directly in the TFDConnection component itself.

Add new comment

Data Connection Backups

david — Fri, 20 Sep 2024 15:24:26 +0000

Data Connection Backups

Category

Programming

david Fri, 09/20/2024 - 08:24

I haven't used Delphi's Data Explorer all that much in years past but in my current job, we maintain hundreds of small accounting programs for dozens of clients, each using a different database and most of which we have backups for testing and support. I had created FireDAC connection definitions for about a dozen of them so I could easily switch between them at design time and view tables and structures from within the IDE. I've found this very convenient and have grown fond of the feature.

I've known that upgrading even minor versions of Delphi uninstalls GetIt packages (which is why I wrote AutoGetIt) but did not know that any Data Explorer definitions would also be lost. After upgrading from 12.1 to 12.2 this morning, I loaded a project group I was working on yesterday and suddenly encountered several Definition [xxx] is not found in [...FDConnectionDefs.ini] error messages.

At first, I panicked, thinking I would have to tediously recreate all those connection definitions. But I noticed the error message listed the file where it expected to find the definitions and figured I should look in that folder to see if there was any way to recover the ones I had established before the upgrade.

Sure enough, I found a few .BAK files that were date/time stamped with past dates, one just a few days earlier. I made a copy of the most recent backup, deleted the new, nearly empty .INI file that was created during the Delphi upgrade, then renamed my copied backup to the default filename as shown in the error messages. When I restarted Delphi, all my database connection definitions were back!

Add new comment

AI Image Generation

david — Fri, 06 Sep 2024 06:20:56 +0000

AI Image Generation

Category

Miscellaneous

david Thu, 09/05/2024 - 23:20

The fascinating realm of AI image generation has opened a new playground intersecting art, human creativity, and serious back-end computing. It's also opened questions of how we will tell real from fiction as this technology evolves and becomes even more sophisticated than it is already. For now, I'm just having some fun in my spare time!

From the vast store of uploaded images and video that grow by the billions every day, high-end computers using machine learning algorithms combined with large language models (LLMs), can turn a simple textual prompt into an image that is often quite realistic. For example, this image was generated with the prompt, "greek roman soldier wearing helmet and glasses sitting at a computer programming" and was generated using ArtSpace.ai. (I'm pretty sure he's a Delphi programmer, don't you?)

Besides using prompts, AI image generation tools also let you base your new image on a theme or model. These next two images were generated using a similar prompt but different models; the one on the right uses one called, "Dream Studio" while the one on the left uses "Nova Flux 2024" (which is ArtSpace's newest and most realistic model).

Another factor at ArtSpace.ai, the tool I used for generating all the images on this page, is the seed. This can either be a number or (on some models), an image. I have not had success using an image to teach it "likeness" yet, I'm still learning the nuances of this technology. What is interesting is how very different images can be generated from the same prompt by simply using a different seed number. The following images were all generated using the same model and prompt but with one digit different in a 10-digit seed number:

AI image generators are not perfect. It can sometimes take several iterations of modifying your prompt to get the image you want; and it may actually lead you down a rabbit hole of exploration in a direction you did not expect. If you have a tool that lets you explore without time or cost limits, that can be a fun distraction. But it can be frustrating if you don't have the time to explore and don't know why the computer can't read your mind! I've seen a lot of example prompts that are long and explicit and generate amazing artwork. Learning to talk so the computer generates the image you want is a fine-tuned craft. It's all too easy to confuse the AI with text that seems innocent and clear to us but causes the generator to place an arm or shadow in an impossible position or not give enough fingers on a hand. I generated one of a woman in a bikini on a boat dock but in one of the iterations added the phrase, "motioning to join her in the water" and the AI added a third arm.

AI image generators can also be used to envision products that don't yet exist or describe a home remodel taking shape or how new furniture would look in your living room. Colors and sizes and shapes can be changed and the image regenerated. I saw a video recently how retailers are using AI to help customers visualize themselves in new clothing or a different pair of glasses. To the left is an image created from the prompt, "large, futuristic dark blue motorcycle with chrome exhaust pipes, sitting in a showroom, oversize rear tire, blurry background". With a several more lines of descriptive text, an engineer could see potential problems with a design before spending hours in a CAD program.

You may have been playing around with some free tools or perhaps you've subscribed to an AI service to help in your programming and it happens to provide an image generator as well. If you are still looking, I invite you to check out ArtSpace.ai which is one I discovered recently. It seems to be pretty good, has a lot of variety, the limit of 500 images a month is plenty for me, it's fairly quick (about 11 seconds per image), and I can download them from the web browser interface and use them any way I want--even commercially! Most software tools these days have a monthly or annual cost to use them and ArtSpace.ai has both; but they also have a seldom-seen one-time-price (which is just a little more than the annual price). For me, it was just too irresistible!

Wow! Thanks for sharing this…

Michael Riley (not verified) Sun, 09/08/2024 - 16:13

Wow! Thanks for sharing this David.

Not directly related to Delphi

david Sun, 09/08/2024 - 18:09

You're welcome, glad you liked it!

I debated whether to write about this or not since it's not directly related to Delphi programming but I've been having fun and as soon as I started generating images, ideas came to mind about how these could be used to enhance blogs, to illustrate ideas, or come up with a cool, new avatar (like the one at the top of this page!).

But so far, it's just for fun; for example, I play board games with some friends and generated this one yesterday:

Add new comment

GExperts - A Most Awesome Delphi Companion

david — Fri, 23 Aug 2024 23:07:41 +0000

GExperts - A Most Awesome Delphi Companion

Category

Programming

david Fri, 08/23/2024 - 16:07

GExperts is one of the oldest and probably the most well-known plug-in for the Delphi IDE. I've been using it since the early versions and still use it in Delphi 12 today. Some of its features have been incorporated into the IDE itself over the years but it continues to evolve and be a useful productivity aid.

In case you don't know what GExperts is, the website describes it as "a free set of tools built to increase the productivity of Delphi and C++Builder programmers by adding several features to the IDE." I agree and would go on to say it has so many features that there must be something beneficial for everyone!

Here is a list, in no particular order, of how I personally have used GExperts to improve my productivity over the last 20+ years. And yes, I have used every one of the features listed below at one time or another; and no, this is NOT all the features!

Code Proofreader

When working in early versions of Delphi (e.g. 5 and 7, 2007), the Code Proofreader not only helped keep identifiers cased consistently but can also save lots of time correcting typos with its AutoCorrect list. How many times have you typed ;= when you intended to type := ? Or "ned" when you meant "end"? GExperts will fix that for you on-the-fly.

But you can expand that idea to use it as a macro replacement, somewhat like a simplified version of today's Live Templates in Delphi. For example, I've added "ef" as an auto-correct with its replacement value set to ":= False;" and "et" with a replacement value of ":= True;". This speeds up initializing Boolean variables significantly. I've added several other similar macros for other common code snippets.

Code Librarian

If you find yourself writing a bunch of programs that use similar techniques, such as accessing a plug-in architecture or an obtuse interface or even just a common SQL statement, and it requires writing similar code over and over, you might find yourself opening old programs or saving snippets of code in a text file you keep close by. But what if that code could be organized in a hierarchical list and built into Delphi, accessed with a few clicks and pasted right in your code with a hotkey? Well, GExperts does just that with its Code Librarian feature. What's more, this is shown in a floating window that can be moved to a second monitor for even more convenience and has several sample snippets to jump-start your collection. It has keyword search and color syntax highlighting. Once you find your code snippet, just click Ctrl+Shift+V to insert that block of code right into the Delphi editor.

ASCII Chart

Have you ever needed to look up the ASCII number for a character? With GExperts installed, simply select ASCII Chart from its menu and you'll instantly have a handy reference to all 256 characters, plus you can select what font and size to see them with.

Backup Project

Even with source control projects in place, I occasionally use the Backup Project feature of GExperts. Usually, this is for copying the project and all the units it uses to a temporary folder or to zip up for some reason, like to send to a friend. This feature saves time by collecting all the units referenced in the DPR of the Delphi project, allowing you to modify the list of files, then creating a .zip file of your choice with the files at the click of a button. That's easier than manually selecting the files in Windows Explorer, isn't it?

Editor Experts

There are several features that are only active while working in the code editor. They use configurable shortcut key combinations that, once learned, add several convenient tricks. Here are three of them:

Expert: Insert Date/Time

One company I worked for required all changes to code to be marked with a commented date accompanied by a "doc header" section at the top of the unit with that date and a comment about what ticket this resolved. It got tedious manually typing something like { 03/15/24 } at the end of every line you where you made changes that day but with GExperts configured to use the date format I need, this became as simple as hitting my configured shortcut key combination, Ctrl+Alt+A!

Expert: Reverse Statement

Have you ever typed up several statements to write configuration settings to an .INI file, then had to retype a bunch more similar ones to read the same settings? Perhaps you've copied, then pasted one set, then manually switched the assignments around. With GExperts, you can save a lot of copy/move/paste operations. Just select the pasted second set of statements and hit Shift+Alt+R (or your assigned shortcut keys) and all those assignment statements are switched around for you. Of course, you'll have to rename the Read statements to Write statements but select the block and use Find/Replace (or better yet, switch to Sync Edit Mode in modern versions of Delphi) to do that.

Expert: Comment/Uncomment

Before Delphi had the comment/uncomment hotkey, GExperts provided this feature, albeit not quite as smart. Instead of a toggle, there was one shortcut to comment (Ctrl+Alt+.) and another to uncomment (Ctrl+Alt+,). It doesn't detect whether or not the lines are already commented, still, this is another nifty time saver when you're in the code in an old version of Delphi.

Replace Components

There have been numerous times when I've wanted to replace one component or set of components with another. Like when changing all buttons on a form to TBitBtn in order to add button icons. You can always manually edit the DFM file and use Find/Replace from one component type to another then acknowledge the changed properties and correct the source's reference when saving the unit but there's a much easier way: select Replace Components from the GExperts menu while one of the button components is selected, select the new component type in the drop-down list of the pop-up window, check the desired Scope (such as "All components on the current form"), then watch as GExperts makes the change for you in a flash!

Components To Code

Most of the time, a component can be simply placed on a form or data module and you can let the framework manage the creation and destruction of the component for you, also allowing you to set properties conveniently at design-time. There have been several instances where I've started down that path, then realized I need to create the component in code, either because I was converting a data module to a component and didn't want to worry about form resources; or when I needed to make multiple copies of the component dynamically.

Setting up all the properties at design time is simple, but can be tedious to do it with code when there are a lot of properties. Here again, GExperts can simplify your life; just select the component you've set up on the form or data module, select Components to Code from the GExperts menu, and it will put all the Delphi statements to create the code on the Clipboard so you can then paste the lines into your code wherever you need.

To Do List

I'm pretty sure that many features of GExperts have inspired features now a standard part of Delphi--and I'm pretty sure that the To Do List is one of those. It's an ingenious but simple idea that if a curly-braced comment is started with TODO:, a list can be generated from comments in all the current source files to provide you a centralized view of tasks. This is another feature with a floating window, and this one can be docked or moved to a second monitor. It functions nearly identical to Delphi's TODO list in modern versions of the IDE.

Message Dialog Wizard

Have you ever added a MessageDlg statement in your code and then at runtime realized it didn't look quite right or you inadvertently switched the Caption with the Title? With GExperts Message Dialog wizard, not only can you build your message dialog statement visually but you can also preview it before a line of code is written! But wait there's more--you can also switch to using MessageBox instead or opt to embed the whole thing in an IF or CASE statement that checks its result! How convenient is that?

Set Tab Order

If you have several controls on a form and add and/or rearrange them over time without setting their tab order, running the program at runtime can be frustrating for a user who tries to navigate among them using the Tab key. Paying attention to the tab order of the various components can increase usability significantly. While new versions of Delphi have a feature to reset the ordering for you, the one in GExperts has more features--and has been around a very long time.

Configuration

There are some parts of GExperts that don't write code or provide real-time functionality but simply improve Delphi's IDE in subtle ways. Here are a couple of areas I'd like to point out.

Config: Component Palette

If you use an early version of Delphi and have lots of components on your horizontally positioned component palette, it can be difficult to find the right tab for the component set you're looking for because you have to slowly scroll through the component tabs until you find the right one. GExperts provides an option to add tabs on multiple lines so you can see all the component tabs at once. Of course, this takes more vertical space so you have to adjust (and save) the editor layout but this is a huge convenience when working with forms and data modules in old versions of Delphi with lots of third-party components installed. If you don't like multi-line component tabs, GExperts has another option to add a pop-up window with all the tabs listed (and alphabetically, if you like); no more component tab scrolling! (This feature is not needed in newer versions of Delphi and was removed from GExperts.)

Config: IDE

GExperts has grown and added features over the years to accommodate new capabilities of Delphi. The IDE tab of the Configuration is one area that has changed a lot; some of the new options are simple niceties, like "Enhance IDE dialogs" that lets you resize and save the position of some of the fixed dialogs.

Others can save lots of time, like one I've recently discovered, "Enhance Build Events dialog with Favorites". This is a huge time-saver for me because I use Build Events a lot and often copy them from project to project. Before finding this feature, I would often close the current project, open an old project that used the build event I needed, go into Project Properties > Build Events and copy the command, then close and reopen the project I was working on and go back into Build Events to paste the command. Now, if I think I'll ever use a Build Event command again, I can just add it to my Favorites list and then select it later when I'm working on a different project, saving me many steps. This has become another of my new favorite productivity boosters!

Look through these options and the other Configuration tabs--you may find some features I have not mentioned that work to solve a tedious task for you!

Copy Your Configurations

Let's say you've added several Code Librarian snippets, added several Dictionary Words to the Code Proofreader, and maybe created a few entries in Favorite Files (another feature of GExperts but not mentioned here), and now you are upgrading Delphi or using a different version of Delphi to work on another project and you want all your GExperts settings you've taken time to customize. The settings for GExperts have had very few changes over the years and are (mostly) compatible with each other (and when the format changes, there's usually an automatic conversion). This means you could copy the configuration files GExperts uses from one installation to another in order to keep your settings.

These configuration files are not a secret. Just look in GExperts Configuration; in old versions, go to the File Locations tab for the directories; in newer versions, go to the General tab and the folders will be shown in the File Locations group box. These folders can be changed if you need; in fact, you may experiment with putting them on a network or synchronized folder.

Closing & Acknowledgements

There are many more features of GExperts not mentioned here, like the Class Browser, Clipboard History, PE Information, Component Grid, and especially several Editor Experts I have not taken the time to learn. Look through the menus, read the documentation, and use the capabilities that you find most useful. There may be some tricks you had not thought of that simplify your coding experience in surprising ways. Some features have been removed from GExperts over the years as the IDE has changed and no longer supports functionality in the same way; others have identical functionality in the IDE so I no longer use the GExperts version of that feature. Still, I use old versions of Delphi enough, and there are still enough productivity gems for the most recent version of Delphi, that this plug-in is still the most valuable Delphi plug-in for me.

I would like to thank Thomas Müller for continuing the support of GExperts after the original programmer, Erik Berry, stopped supporting the project a few years ago. It's an open-source project and you can contribute suggestions and ask questions on Delphi PRAXiS forum dedicated to this product.

See the Webinar

Finally, there was a webinar on GExperts by Jim McKeeth of GDK Software that demonstrated many of these features--and a few others I have not used. They will be presenting other popular Delphi plug-ins in the near future, so be sure and visit their "webinars page" to learn more.

Totally Agree.

Ian (not verified) Fri, 08/23/2024 - 19:16

Absolutely concur. It is always the first Plug-In I install..

Add new comment

Take Control of Your Delphi Project

david — Sat, 22 Jun 2024 02:06:03 +0000

Take Control of Your Delphi Project

Category

david Fri, 06/21/2024 - 19:06

You can (and should) have complete control over all aspects of the version information, application icon, operating system compatibility, requested user access, Windows theming dependencies, and more in your Delphi projects!

For the better part of a week, I struggled to get some of these aspects configured properly for a suite of programs that are all configured similarly and for which I was building a script to migrate and manage. I had to learn about Microsoft Manifest files and how they are linked into Delphi projects, where to list a common application icon and get it to show up in the Windows TaskBar consistently regardless of individual form settings, and how to manage version info centrally for these programs needing common information such as company and copyright but also per-project data like product name and version number. I've been programming for a long time in Delphi but had just been letting Delphi manage most of this information--sometimes with frustrating consequences.

It was past time to take control!

This blog lists three different types of resources that can be linked into your Windows application. It serves as both a reminder to my future self and a tutorial for others and pulls together a couple weeks' worth of research, trial and error, and scouring the web for bits of information.

APPLICATION ICON

In many VCL programs, simply selecting and assigning an .ICO file to the main form's Icon property, is all you need to set the icon of the application and to see it show up on the TaskBar when running. There are also project properties to set an icon but a main form's icon will override that; if your main form does not have an icon set, the project's icon will show on both the form and the TaskBar.

However, if you need to override any defaults set by the project or a form, you can set the Application.Icon property in code at runtime. The two simplest ways to assign the icon is to call one of the Icon property's methods, such as LoadFromFile or LoadFromResourceName. The latter requires the icon to be stored either in the .EXE or in a loaded resource file such as a DLL. In my applications, I want to store the icon in the EXE itself to keep things simple. A convenient way to do that is to select Project > Resources and Images... from the Delphi menu, click Add, then select your .ICO file, set the Resource Type to ICON, and give it a good Resource Identifier (I chose "APPICON").

Now, when your application is running simply pass in HInstance (the handle to your application) and 'APPICON' (or the name you chose for your Resource Identifier) to override the default icon:

Application.Icon.LoadFromResourceName(HInstatnce, 'APPICON');

The inclusion of this icon as a resource in your executable is recorded in the .DPROJ file for your project and the icon filename shows in the IDE's project manager alongside your code units; it triggers the resource compiler at compile time to generate and build a special resource file with the same base name as your project and with a ".dres" extension. When resources are added to your project like this, a line is also added to the .DPR file for your project:

{$R *.dres}

VERSION INFO

There have been several blogs over the years mentioning the poor version management in the Delphi IDE. The advice is to never allow Delphi to control the version information but to always manage it yourself externally. How do you do this? First, in the project options, make sure "Include version information in project" is unchecked for all configurations. Then, create a resource source file (.rc) with your version information specified in the right format, call the resource compiler to build this resource, and add a $R compiler directive to link it in.

That may sound like a lot of work but I would invite you to read Thomas Müller's blog and download his dzPrepBuild utility. Not only will it build your resource file in the right format for you but it is built to be used in automation scripts to keep it updated with incrementing version numbers and copyright years.

Following the guidelines for using this tool, my version info resource filename was <ProjectName>_Version.rc. Thus, I added a Pre-Build event to the project to build the resource just before compilation:
"$(BDS)\bin\brcc32.exe" $(OUTPUTNAME)_Version.rc

Finally, I added one line to the project's .DPR:
{$R *_Version.RES}

Correction--I actually replaced one line in my project's .DPR: I replaced the {$R *.RES}, incorrectly thinking I no longer needed that line. My belief was that since I cut out the automatic version info and replaced it with my own manually built version info resource, this line was no longer needed. However, that's when I noticed a visual change in my VCL applications and started studying manifest files.

MANIFEST

I had spent a bunch of time researching resources and version info and setting up a script and was feeling pretty good about how projects were being upgraded from Delphi 5 to Delphi 12 with new, more standardized configurations and a modern look and feel. I had been showing progress to my colleagues each week and how much time we would be saving. Then, suddenly, I noticed a visual change in the toolbar of all the programs that had been compiled with these new automated scripts and linked resources.

We're using the RzToolbar from the Konopka Signature VCL Controls package and set a gradient style on the toolbar to go from clBtnFace on the left to clBtnShadow on the right. With all the buttons set to transparent, the icons floated on the toolbar with no borders until you hover your mouse over them, giving a nice, clean and responsive look to the application. But now, the buttons had lost their transparency--all the buttons had a white-ish box, the ones on the far left closely matched the toolbar color but it was very obvious on the right side.

I tried everything I could think of, double-checking the Transparency property of the buttons, checking and unchecking "Enable Runtime Themes" in the manifest section of Delphi's project options, changing DPI awareness, looking through form properties--all to no avail. My research led to a couple of articles about manifest management in Delphi and, like project management, read that I should create my own manifest file--the blog by Vincent Parrett was especially helpful with explanations and examples.

I had written a sample program to test the gradient toolbar and set the project properties to use a custom manifest file that I gleaned from the examples and modified. After lots of trial-and-error testing I finally got to the point where I could break, then fix the transparency issue with the toolbar buttons by simply changing a section of the XML in the manifest. By this time, I had learned the manifest is treated like other resources and linked in--and it uses the default project resource name which I had thought was only for version information. By simply adding that default {$R *.RES} line back in the .DPR, everything was working again. (I wish the documentation on Delphi's support for manifest files had mentioned that.)

So, what was the key to getting transparency working in the program? It turns out that enabling support for Windows Themes (which is what this transparency feature needed) was simply requiring a dependency on Microsoft Common Controls so that it would use a newer version of the visual controls. This is mentioned on an obscure page in the DocWiki where it shows the default manifest provided by Delphi for Windows targets. I would not have had to create a custom manifest if I had simply left that $R directive in place; however, I now know a lot more about what you can do with a manifest file by having read through the Microsoft documentation on Application Manifests. For example, you can require admin privileges for a program (in <requestedPrivileges>) and you can require Windows 10 or above (in <compatibility>); there are other fine-grained constraints you can add to your applications if you need by crafting a custom manifest.

Sample Project

To preserve my research and make permanent use of the sample program that helped me figure this out, I put it up on Github. It's called RzToolbarTest and includes a project file with associated icon, a resource source (.rc) file to specify the version info, and a manifest file, all linked in using three $R directives.

I hope to save you--and my future self--some time with better control of Delphi projects!

Wow, somebody else is…

Thomas Müller (not verified) Sat, 06/22/2024 - 02:05

Wow, somebody else is actually using my dzPrepBuild. I was under the impression that I was the only one.

@Thomas Müller : I think…

Dan (not verified) Fri, 07/12/2024 - 11:34

@Thomas Müller : I think congratulations are in order :-)

Recently Found

david Sat, 06/22/2024 - 06:42

I only found it a couple of weeks ago. It all started from a conversation on Delphi PRAXiS when Jim McKeeth mentioned the reFind utility which got me started down the path of automating Delphi project migration. I had a large script filled with regular expressions and it was going well but there was a hole in the process that required manual editing after converting a Delphi 5 project (with no .dproj) up to Delphi 12. I didn't want to rely on programmers setting up the version info manually as it would surely result in inconsistencies. Besides, it's just text and the script had so much of the project information available already--surely that could be automated as well? So, I went looking for a tool to fill that gap and found the perfect solution with yours! And it will also fit in nicely with plans to set up a build server. I just hadn't gotten around to mentioning it on the forum yet because I just finished it late yesterday. Thanks for writing that utility--and blogging about it!

Add new comment

Delphi as a Build Machine

david — Thu, 09 May 2024 02:07:58 +0000

Delphi as a Build Machine

Category

Programming

david Wed, 05/08/2024 - 19:07

Building software projects involves, in addition to writing code, repetitive tasks like compiling, testing, signing, and deploying your completed application. Many times, a complicated suite of programs needs to be built in a certain order, copied to multiple locations, or even set up with special parameters for various customers. Large teams will likely have a dedicated DevOps person to handle these tasks, often with special "build machines" and complex scripts. Many development groups will utilize automation software to accomplish these tasks on a continual basis.

As a Delphi developer, perhaps your organization has invested in FinalBuilder or Project Maker or maybe they set up a build server with Jenkins or even figured out how to get a self-hosted runner for GitHub working. These all take time and effort and a little money to establish; and while they pay off in the long-run, they may be just out of reach for your skillset or budget, or simply overkill for the small projects you're working on.

But that doesn't mean you're stuck doing everything manually. Did you know you can perform many aspects of build automation right within the Delphi IDE on your own desktop? This is accomplished with three built-in features that work in conjunction with each other:

Build Events,
Project Groups, and
Build Groups.

Let's discuss each of these.

Build Events

A "build event" allows you to execute custom commands automatically as part of your build process. This can include tasks such as copying files, running scripts, or invoking external tools. There are two types of Delphi build events, pre-build and post-build events. As their names imply, pre-build events are commands that are run just before the project is built and post-build events run immediately afterwards. You can set flags so that errors returned from these commands halt further processing. For example, if a pre-build event tries to get the latest version of the code from a server but the server is unavailable, the build could be halted so you're not building something that might be out of date. Similarly, a post-build command could run a series of tests on the compiled project and if it fails, prevent further actions from taking place so that you can address the problem before sending out a faulty application. You can also define different build events for different platforms or configurations. For example, the debug executable might be copied to a network share for others to test while the release might get uploaded to a website. I often use build events to apply code-signing to release versions of executables.

To configure build events in Delphi, right+click on your project in the Project Manager, select Options, then select Build Events under the "Building" section. You may also see "Pre-link events"; those are only applicable to C++ projects.

Build event commands are executed by the IDE as if they were run on the DOS command line. You can add more than one command but they're stored concatenated with an ampersand (&) in the project file which, in some older versions of Delphi are not handled properly (it also may cause parsing problems if you use an ampersand in your command). Many programmers write batch scripts that are called by the build event if they need to execute more than one command during a build event.

One very nice feature of using Delphi's build events is having access to pre-defined macros in the IDE. For example, $(OUTPUTDIR) gets expanded to the folder where the compiled project will be placed and $(Platform) is the project's active platform name. These macros get replaced by their character values when the command is executed and can be used directly or passed in as a parameter to a script called by the build event. The build event editor shows the list of available macros in the bottom half of the window.

In the screenshot above, I call a signing tool to code-sign the release version of a compiled program and pass in $(OUTPUTPATH), which gets replaced with the completed executable's full path and filename when executed. If I change the project name or the path where compiled executables are placed, this build event command still works; also, I can copy this command to a different project and not have to change anything because I used a macro instead of hard-coding the filename. Note that I enclosed the macro in double-quotes because the path may include spaces or special characters that would cause parsing problems when run on the command-line.

Debugging build events can be a little tricky; it's best to test the commands manually on the command line before putting them into a build script. When executed by a build process in Delphi, they appear in the "Output" tab of the "Messages" pane with their macros expanded.

These automation capabilities really become useful when several projects are grouped together into a Project Group.

Project Groups

One of the oldest features of Delphi's project management is the ability to group several projects together into a Project Group. A project group allows you to manage multiple related projects together without needing to unload one project, then load another. These projects can be sub-modules of a main application (e.g. DLLs or BPLs) or related programs possibly launched by the main application. In any case, it is useful to navigate between them when they're all visible in the same workspace.

To create a project group in Delphi, simply load a first project like usual, then add an existing project to the group by right+clicking on the project group node in the Project Manager and selecting "Add Existing Project." You can add several projects to the list by repeating this process. You can then right+click on the project group name and change the name, compile or build all the projects with a simple mouse click.

When you select to compile or build all projects in this manner, the order of the projects can be important as compilation order of a "Build All" process always starts with the first project at the top and goes sequentially down the list. You may have projects that depend on others being compiled and available before they will successfully compile. You can adjust the order of the projects in the project group by right+clicking on a project and selecting either "Build Sooner" or "Build Later" from the pop-up menu to move the project up or down in the list.

The buttons at the top of the Project Manager provide options for managing your project group. In addition to the convenience of having them lumped together for quick navigation, you can also change the configuration or platform of all the projects in the group at once.

One of the Project Manager buttons takes this convenience and functionality one step further, it activates the Build Groups pane.

Build Groups

When I first noticed Build Groups, I thought it was just another way to compile all the projects and so ignored it for quite some time. In one sense it is but after seeing a couple of articles on it a while back and exploring it further on my own, I realized there's additional value here that can take a developer's build process automation to the next level without leaving the IDE.

Basically, a build group allows more flexibility when building all projects in a project group by allowing you to specify which projects to build; but more than that, it also allows you to select multiple configurations and platforms to use during a build process, meaning it can compile each project multiple times in one operation!

Build Groups can compile each project multiple times in one operation!

Let's say you need to build your suite of applications and modules for three different platforms; Build Groups can do that with one button click. Let's say you need to build all but one of your projects but you don't want to remove the project from the group; you can unselect any of the projects from the configured Build Group.

When you activate the Build Groups pane, it shows all your projects and any defined build groups in a drop-down; initially there are no build groups but you can add, copy, and delete them with the toolbar buttons to the right of the list. When you add a build group, you give it a name and it creates a list of all the projects in your project group selected and with the current configuration and platform for each. You can then click in the space next to the configuration or platform for each listed project and click the button to add or remove various configurations or platforms.

In the screenshot below, I've selected all projects in my project group, Debug and Release configurations for all but one of the projects, and various platforms for the different projects.

Once your build group is configured to your liking, you can click the appropriate button on the Build Groups toolbar to start the compile or build process, along with any defined build events for each selected project, then sit back and watch your computer work. If all goes well, clicking the Compile or Build buttons for this build group will generate 20 executables!

Conclusion

While this approach may not offer the sophistication of dedicated build automation tools, it can significantly reduce manual effort and ensure consistency in the build process. So, whether you're a lone coder or part of a team, consider harnessing the power of Delphi's Build Events and Build Groups to create a basic build automation system on your own desktop. By organizing projects effectively and automating repetitive tasks, you can streamline your development workflow to build better software, even on a shoestring budget.

Add new comment

Flexibility With Runtime Packages

david — Tue, 23 Jan 2024 08:38:13 +0000

Flexibility With Runtime Packages

Category

Programming

david Tue, 01/23/2024 - 00:38

Delphi's project options allows you to build a project with or without linking to runtime packages. A runtime package is a special dynamic-link library in the proprietary Delphi (or C++Builder) format that allows an executable to be split into various modules of functionality; these module files have a .BPL extension (for Borland Package Library). In the early days, this was useful to allow large applications to run on computers with limited memory because each module loads into its own memory space (these days, memory limitations are rare unless you're still deploying memory-hungry 32-bit applications). The main advantage today is realized if you have multiple executables that share some common functionality, like reporting or GUI controls or database access. The .EXEs can be smaller because instead of linking the reporting or GUI controls or database access code into each one of the .EXEs, you can ship that functionality in .BPLs and the .EXEs can be much smaller because they load that shared common code at runtime.

To change this option, pull up your Project Options and find the "Link with runtime packages" option. If that option is not checked, you're building a monolithic executable--all the library code is in one big .EXE. But if you check that box and list the runtime packages for your applicaiton, your .EXE is smaller and you must ship the specified package files along with it. To be automatically found by the .EXE, the .BPLs must be either in the same folder as the .EXE or on the system PATH.

What I didn't know is that you don't have to list all the runtime packages your application uses!

I've known about this feature for many years and have used it at times for specific scenarios, or when a working with a plug-in system that used the .BPL format. What I didn't know, however, is that you don't have to list all the runtime packages for that option--and what packages you DON'T list, get linked into the .EXE. In fact, if you don't list ANY runtime packages, Delphi builds the application as if you had not checked the "Link with runtime packages" option at all!

This means that you can determine how many of the packages to link in and which ones to leave out as extra .BPL files. My assumption has always been that it was all or nothing--either all packages were linked into one giant .EXE or all packages used by the application had to be listed and deployed for it to work. This is not so!

By default, the "Link with runtime packages" option is unchecked for new application projects, allowing you to quickly create a stand-alone program. When you first check that option and enable the "Runtime packages" line, all the packages for all components you have installed in Delphi, whether from Embarcadero or open source or commercial products, and both VCL and Firemonkey, are all listed. You obviously don't need all these packages and cannot even build a project with everyone of them listed as they are not all available for all platforms; for example VCL and Firemonkey are incompatible in the same project and the BDE doesn't work on any platform other than Win32.

If you're going this route with runtime packages and are deciding which ones to include, it may be prudent to stop periodically and build your project to see how big the .EXE size is. Depending on how many packages and .EXEs you have to ship, there will be a "sweet spot" that makes sense for you. A team I'm working with deploys different sets of several dozen .EXEs to hundreds of customers and while there could be an argument to save hard disk space, it's so cheap these days, it's hardly worth the time. What's more important for us is if we want to to send out a fix to a reporting library or upgrade the security of one of the internet components to support a new protocol or change a style of a GUI control. By keeping some of those types of .BPLs separate, we can update just a few files to affect all programs rather than recompile and redistribute hundreds of applications.

After learning about this flexibility in listing runtime packages, I wanted to test it out and see some real numbers. So, I wrote a simple Win32 VCL application in Delphi 10.4 and placed a label, a button, and two third-party components on the main form, 1) a ReportBuilder component, and 2) a non-visual layout saver from a small library I wrote. I built this four different ways, noting the size of the .EXE and the list of packages to deploy for each scenario:

OPTION: NOT linked with runtime packages
EXE Size: 26,690 KB
Deploy: single .EXE

OPTION: Linked with runtime packages: only ccLib_R
EXE Size: 14,581 KB
Deploy: .EXE plus...

rtl270.bpl
vcl270.bpl
dbrtl270.bpl
cclib_r270.bpl

OPTION: Linked with runtime packages, only dclRB2227 (ReportBuilder)
EXE Size: 4,224 KB
Deploy: .EXE plus...

rtl270.bpl
vcl270.bpl
vclx270.bpl
vclWinX270.bpl
vclimg270.bpl
vclie270.bpl
vcledge270.bpl
bindengine270.bpl
rbRTL2227.bpl
rbRCL2227.bpl

OPTION: Linked with runtime packages, both dclRB227 and ccLib_R
EXE Size: 4,210 KB
Deploy: .EXE plus...

rtl270.bpl
vcl270.bpl
vclx270.bpl
vclWinX270.bpl
vclimg270.bpl
vclie270.bpl
vcledge270.bpl
bindengine270.bpl
rbRTL2227.bpl
rbRCL2227.bpl
ccLib_R.bpl

Several things I noticed in this test: 1) removing my small library from the stand-alone .EXE dropped its size by quite a bit but only because it also separated out the RTL and VCL packages that had been previously linked in; 2) simply listing a single ReportBuilder package required eight additional .BPLs to also be shipped; 3) the real size of the "ccLib" package was realized in the last test as it only reduced the .EXE size by 14 KB from the previous test.

A final consideration when selecting your package list is the hassle to package and deploy the various .BPL files; is it worth the extra management required to make sure all the .BPLs are deployed in order to save a few MB? The questions you have to answer will be different for your software and customer environment. But with modern build tools and automation, a lot of the details can be tucked away, never seen or dealt with after the configuration is tested so that is probably not a big deal.

Well, if you didn't know this trick about varying the selected packages to include before, you now have a little extra flexibility for how you build your Delphi projects.

Add new comment

First Drupal-to-Hugo Conversion

david — Fri, 05 Jan 2024 07:33:01 +0000

First Drupal-to-Hugo Conversion

Category

Cloud Computing

david Thu, 01/04/2024 - 23:33

Back in May of 2022, I wrote about a predicament I found myself in with multiple websites built in Drupal and the chore of upgrading in the right sequence to avoid incompatibilities with the installed version of PHP on the web server. I began to look around at my options and discovered others that also had fallen out of love with Drupal were turning to static web page builders, several of them to Hugo.

This sounded very much like the adage, "What's Old is New Again!" But this time it's different.

In the beginning, we all built our sites by hand with HTML. It wasn't long before that became quite tedious and we turned to scripting to add content to our sites, using such languages as Perl or PHP or maybe ASP.NET. This was, of course, much better than manually updating HTML pages, especially if you also maintained a table of contents, an index, and a site map. HTML editing tools proliferated.

Then some people productized their scripting and Content Management Systems became a thing. This was even better because it allowed non-programmers to maintain their site and encouraged more features and multiple people and separation of code and data and design. All these were good things but as web sites got more complex and security issues crept in, these simple frameworks blossomed in size. Some of my early Drupal sites were less than 500 KB; the latest version of Drupal is nearly 120 MB and includes almost 20,000 files--and that's just the core install. After that, there are modules and themes and profiles and customizations--then security updates and bug fixes! IT NEVER STOPS!

So, now I want to throw all that away and go back to hand-editing my content? Well, yes, actually!

My research and testing led me to try out Hugo; a year ago I wrote about this new direction and during 2023, built a couple of small sites in Hugo for a client, Modular Building Solutions, and a personal photo album for our family. I wrote some utilities and really liked editing the simple Markdown pages, running Hugo to rebuild the HTML pages, then uploading. In fact, with GitHub Actions, simply pushing updates to a remote repository can kick off an automatic site update.

I was having fun again!

Now the task of converting the content in my Drupal sites to Markdown pages lay in front of me. The first one I wanted to tackle was the Oregon Delphi User Group for which I am the coordinator. It's version of Drupal was fairly old and there were a couple of minor issues whenever I added meeting notices that I had to work around. Plus, I didn't really like the theme but didn't feel like customizing it. You can see what it looked like on the Wayback Machine.

So, over the holidays, I took some time, found a script (or was it this one?), exported the content, and got to work.

And yes, it took some work to get all the content converted because the Drupal content editor allows different formats of text, including Full HTML, which I often used so I could customize parts of the entries or attach images and align them properly. The problem with that is it doesn't just store the text, it stores the HTML that I entered as well. So what I exported had to have its HTML tags removed and replaced with Markdown. There was probably an easier or more automated way but I had other edits to make as well. Instead of using Drupal's "Taxonomy" system to set up events by year, I just associated "year" tags to the various posts. Then, since I was adding tags anyway, I added a few other tags for speakers and topics. As I kept building and reviewing the finished site (which is nearly instantaneous with Hugo's in-memory server feature), I made other tweaks. Adding a theme was just as fun because I found one that was easily modified to make the posts look like Pascal procedures!

I unveiled it yesterday and so far, I've gotten positive feedback. I think it looks better than the old one, and it's MUCH faster because it's no longer running through a PHP interpreter to generate HTML, it just serves the pre-built HTML files. There's so much less to configure and, of course, there are NEVER any security concerns--but then again, it's just a static site that gets updated about once a month.

Check it out: ODUG.ORG. And if you maintain a static site, you should try Hugo for yourself!

Good move. I also use Hugo…

Andre (not verified) Fri, 01/05/2024 - 17:40

Good move. I also use Hugo and it feels good to have full control of your website. The website is also so lightweight and loads extreme fast compared to all bloated websites.

Nice work and the website looks great 👍

Thanks!

david Fri, 01/05/2024 - 17:43

Thanks!

David, this is a fascinating…

Michael Riley (not verified) Wed, 01/10/2024 - 03:44

David, this is a fascinating article. After reading this I will definitely look into using Hugo.

Add new comment

ChatGPT can write Delphi code

david — Fri, 29 Dec 2023 17:46:55 +0000

ChatGPT can write Delphi code

Category

Programming

david Fri, 12/29/2023 - 09:46

The company I work for has hundreds of small accounting programs written many years ago in Delphi 5 where they have been maintained ever since. As a new member of the team, my primary task is to help them upgrade to newer versions of Delphi. But I also assist in supporting their legacy code for dozens of county offices that rely on them every day. Sometimes, these customers need features that, while are simple to implement in newer versions of Delphi, were not yet added to the toolset in the late 1990s. One such request came through a few months ago and I was surprised to discover that it had never been requested--nor did the rest of the team know how to handle it.

The need was to be able to import data through a SOAP webservice. I didn't learn about this until another programmer had the XML packet, complete with SOAP header he had pulled from the example sent to him, all ready packaged and ready to hand off to some other function. I knew I would be writing a DLL in Delphi 11 that would be called by his Delphi 5 application but I had never called a SOAP service from Delphi with raw XML--I had always used its WSDL importer, filled in properties and called a method of the class. As I looked further at this, trying to untangle what the end result of what I needed to do and thinking it couldn't be that hard, a thought struck me: why not try this new AI tool, ChatGPT, that others have said had been helping them with coding? I already had a free account and had used it a few times for research and to help explain a few technical things but this was a chance to see if it could actually help me save some time as a professional programmer.

So I opened up a new ChatGPT session and typed in:

How do I call a SOAP web service from Delphi without using the WSDL importer?

The response was a great explanation of the process:

Calling a SOAP web service from Delphi without using the WSDL importer involves manually crafting the SOAP request and handling the SOAP response using the Indy components (TIdHTTP) or other HTTP libraries. This approach requires a good understanding of SOAP XML structure and the HTTP protocol. Here's a basic outline of the steps you need to follow:

Create the SOAP Request XML: Manually construct the SOAP request XML using the appropriate SOAP envelope, headers, and body for your web service method. The structure of the XML depends on the specific web service you're interacting with.

Set Up the HTTP Request: Use Delphi's HTTP library (e.g., TIdHTTP from Indy) to create an HTTP POST request to the web service endpoint. Set the appropriate headers, such as "Content-Type" and "SOAPAction" header.

Send the SOAP Request: Send the SOAP request XML as the content of the HTTP POST request. Make sure to encode the request XML properly.

Receive and Process the Response: Receive the HTTP response from the server. Extract the SOAP response XML from the response body.

Parse the SOAP Response: Manually parse the SOAP response XML to extract the relevant data. This may involve using XML parsing libraries like Delphi's built-in XML components or other third-party XML parsing libraries.

Handle Errors: Implement error handling logic to deal with various possible error scenarios, both at the HTTP level and the SOAP level.

Here's a simplified example of what the code might look like using Indy's TIdHTTP component:
uses IdHTTP, IdSSLOpenSSL;
procedure CallSoapWebService; var HTTPClient: TIdHTTP; RequestXML: TStringStream; ResponseXML: string; EndpointURL: string; begin HTTPClient := TIdHTTP.Create(nil); RequestXML := TStringStream.Create('YourSOAPRequestXMLHere', TEncoding.UTF8); EndpointURL := 'YourWebServiceEndpointURLHere'; try HTTPClient.Request.ContentType := 'text/xml;charset=UTF-8'; HTTPClient.Request.CustomHeaders.Add('SOAPAction: YourSOAPActionHere'); ResponseXML := HTTPClient.Post(EndpointURL, RequestXML); // Process the SOAP response XML here except on E: Exception do begin // Handle exceptions end; end; HTTPClient.Free; RequestXML.Free; end;
Remember, this is a very basic example and may not cover all the complexities of real-world SOAP interactions. Additionally, manually dealing with SOAP can be error-prone and time-consuming, so using the WSDL importer is generally recommended for more robust and maintainable code.

I was very impressed but since I'm using Delphi 11, I wanted to know if I could use the newer internet client components instead of Indy; so, I typed this in next:

Can I do this with NetHTTPClient instead?

The sample code was reworked to use this component instead and even acknowledged it provides an easier and more modern way to work with HTTP requests and responses.

After playing around with the updated code for a bit, I noticed it had used the property DefaultHeaders which is not part of the HTTPClient component. I asked ChatGPT about that and it immediately apologized and reworked the code using CustomHeaders instead (why it couldn't have used the correct property in the first place is a mystery to me):

uses
  System.Net.HttpClient, System.Net.URLClient, System.Classes;

procedure CallSoapWebService;
var
  HTTPClient: TNetHTTPClient;
  RequestContent: TStringStream;
  ResponseContent: string;
  EndpointURL: string;
begin
  HTTPClient := TNetHTTPClient.Create(nil);
  RequestContent := TStringStream.Create('YourSOAPRequestXMLHere', TEncoding.UTF8);
  EndpointURL := 'YourWebServiceEndpointURLHere';

  try
    HTTPClient.CustomHeaders['Content-Type'] := 'text/xml;charset=UTF-8';
    HTTPClient.CustomHeaders['SOAPAction'] := 'YourSOAPActionHere';

    ResponseContent := HTTPClient.Post(EndpointURL, RequestContent).ContentAsString;

    // Process the SOAP response XML here

    except
      on E: Exception do
    begin
      // Handle exceptions
    end;
  end;
  HTTPClient.Free;
  RequestContent.Free;
end;

I asked a few more clarifying questions and got really good answers, turned the procedure into a function, added a try/finally block around the bulk of the function to properly free the components, created an interface to convert strings to PChars to work inside the DLL (I suppose I could've asked ChatGPT to do this for me!), and ended up with a useful tool in our library to support our legacy code a little longer.

function CallSOAP(const RequestURL, RequestStr: string; var ResponseStr: string): Boolean;
var
   RequestContent: TStringStream;
begin
   NetHTTPClient := TNetHTTPClient.Create(nil);
   RequestContent := TStringStream.Create(RequestStr, TEncoding.UTF8);
   try
      Result := True;

      try
         NetHTTPClient.CustomHeaders['Content-Type'] := 'text/xml;charset=UTF-8';

         ResponseStr := NetHTTPClient.Post(RequestURL, RequestContent).ContentAsString;
      except
         on E: Exception do begin
            ResponseStr := E.Message;
            Result := False;
         end;
      end;

   finally
      RequestContent.Free;
   end;
end;

This is a really simple example of how AI can enhance programmer productivity.

My experience was a lot worse

Thomas Müller (not verified) Sat, 12/30/2023 - 09:25

I tried something similar several times but always got code that had some compile errors and / or was using some classes or methods that simply didn't exist. chatGPT always aplogized when I told it about these errors and generated new code, that again had the same kind of errors. After a few iterations of that I got bored and wrote the code myself.

And after trying this several times with the same kind of results, I finally gave up.

You seem to have had more luck or maybe your prompts were better than mine.

To be fair...

david Tue, 01/02/2024 - 13:47

To be fair, I encountered a compiler error as well after I switched to NetHTTPClient--it used DefaultHeaders instead of CustomHeaders but it was pretty quick to spot and fix. There may have been a couple other things as well, but the overall experience was that it increased my productivity in this instance by jump-starting the coding process--which is what I was hoping.