Simple background tasks with OtlParallel - Life after 2.1: Async redux

OtlParallel unit defines four overloaded Async methods

Parallel.Async(
   class procedure Async(task: TProc; taskConfig: IOmniTaskConfig = nil); overload;
class procedure Async(task: TOmniTaskDelegate; 
  taskConfig: IOmniTaskConfig = nil); overload;
class procedure Async(task: TProc; onTermination: TProc; 
  taskConfig: IOmniTaskConfig = nil); overload;
class procedure Async(task: TOmniTaskDelegate; onTermination: TProc;
  taskConfig: IOmniTaskConfig = nil); overload;

As it turned out, two of them are not necessary anymore. Since the introduction of the taskConfig parameter, termination procedure can also be specified by setting taskConfig.OnTerminated.

To make the matter simpler, I’ve removed both Asyncs that accept the onTermination parameter.

class procedure Async(task: TProc; taskConfig: IOmniTaskConfig = nil); overload;
class procedure Async(task: TOmniTaskDelegate; 
  taskConfig: IOmniTaskConfig = nil); overload;

The old way of setting termination handler …

Parallel.Async(
    procedure
    begin
      // executed in background thread
      Sleep(500);
      MessageBeep($FFFFFFFF);
    end,
    procedure (const task: IOmniTaskControl)
    begin
      // executed in main thread
      btnAsync.Enabled := true;
    end
    )
    );

… must be replaced with slightly more verbose

Parallel.Async(
    procedure
    begin
      // executed in background thread
      Sleep(500);
      MessageBeep($FFFFFFFF);
    end,
    Parallel.TaskConfig.OnTerminated(
      procedure (const task: IOmniTaskControl)
      begin
        // executed in main thread
        btnAsync.Enabled := true;
      end
    )
  );

Source : Here

FastMM – Preparing your apps to report memory leaks

One of the most challenging parts of inheriting a legacy project is to fix the memory leaks that most often are hiding in the code. A while ago, while dealing with an application that managed to eat all the available memory within a few hours I found FastMM. And it sure was a great find.

What is FastMM?

FastMM is a memory manager replacement designed to be used with Delphi and C++ Builder. It is an Open Source project developed by Pierre Le Riche in South Africa.

Starting with Delphi 2006 FastMM replaced the Borland memory manager. Unfortunately, Delphi only ships with a subset of FastMM. Most of the useful debugging reporting that can be done with FastMM has been stripped from the shipping version of Delphi, RAD Studio and BDS.

But fear not! It is very simple to replace the stripped down version of FastMM with the full version. All you have to do is follow the directions outlined below. I have also included a step to install the FastMM4 Options Interface program. That program is a very friendly way of configuring the options contained in the file FastMM4Options.inc.

How to install FastMM

Download the latest FastMM source code from sourceforge.net or GitHub
Copy the content of the downloaded zip file to a folder on your computer.
In Delphi add a path in Tools>Options>Library – Win32 Library Path to the FastMM folder that contains the unit FastMM4.pas.
Copy the file FastMM_FullDebugMode.dll from the folder FastMM\FullDebugMode DLL\Precompiled to the Delphi install folder. For example in Delphi 2007 – C:\Program Files\CodeGear\RAD Studio\5.0\bin or in the Delphi XE install folder – C:\Program Files\Embarcadero\RAD Studio\8.0\bin
Optionally one can download and install the FastMM4 Options Interface program from JED software’s web site at jedqc
Configuring FastMM

Once you have properly installed FastMM4 you will be able to detect any memory leaks and attempts to use freed memory. But before we do that let’s take a look at what we have just installed.

In the FastMM folder you will find the file FastMM4Options.inc. This is the file that controls how FastMM behaves. Each option is very well documented and it is how you set the default behavior of FastMM. One can manually edit this file or, optionally, use the FastMM4 Options Interface.


Let’s take a quick look to this file. There are eight different sections in this file.

1 – Miscellaneous Options

This section contain general settings to control memory alignment, use of fastMove library, multithreaded behavior and debug only when running the ID

2 – Debugging Options

This section contains defines that control the debugging behavior of FastMM such as logging errors to a log file, dumping of memory along with an error, stack traces and more

3 – Memory Leak Reporting

This section controls the reporting of memory leaks, how to deal with expected memory leaks and the presence of the IDE or debug info to report errors.

4 – Instruction Set Options

This sections deals with using MMX instructions and this option currently only affects the variable size move routines.

5 – Memory Manager Sharing Options

This section allows sharing of the memory manager between a main application and DLLs.

6 – Option Grouping

Allows you to group a set of options for the release version and the debug version of your applications. So you can have the memory manager report issues when your application has been compiled for debugging and quietly ignore memory errors when compiled for release.

7 – Compilation Options For borlndmm.dll

If you’re compiling the replacement borlndmm.dll, set the defines in this section for the kind of DLL you require.

8 – Patch BCB Terminate

To enable the patching for BCB to make uninstallation and leak reporting.

What if I don’t want to mess with FastMM4Options.inc file?

Of course, you don’t have to edit the file manually. You can use FastMM4 Options Interface. However, you will not have access to the compilations options for debug and release version of your programs. Nevertheless, it is a very user friendly straightforward way of making changes to the default behavior of your application.

Switches are grouped in different tabs and for each option an detailed explanation (from FastMM4Options.inc) is displayed on the right pane.

Once you start the program you need to load the appropriate Option file, make the desired changes. Save the changes and rebuild your application.

Preparing and compiling projects in Delphi

There are a few simple steps to prepare your existing projects to use the full version of FastMM in a useful way.

You can control how FastMM behaves in two ways:
By making changes to Option Grouping in the FastMM4Options.inc so you can define different FastMM behaviors for your release version and debug version.
Or by using a conditional IFDEF statement to use the complete library on the debug version and ship the Delphi supplied library in the release version. To accomplish this, in Delphi open a project and add the unit FastMM4 as the first unit in the uses clause of the .dpr file.

uses
{$IFDEF DEBUG}
FastMM4,
{$ENDIF}
//DB,

Optionally, set FatsMM4 debug options using the FastMM4 Options Interface program. Make sure to build your program every time you make changes to any FastMM4 options.
In order for certain debug features of FastMM4 to work you must make sure that certain debug switches are turned on. The following is a list of recommended switches:
In the Compiler options set the following options
Debug Information
Reference Info
Use Debug DCUs
In the Linker options make sure that oneof the following options is set
TD32 Debug info
Map file
After you run the application a log of the memory manager can be found in the same folder where the application ran. The log file is named Leaks_MemoryManager_EventLog.txt.
If your project includes EXEs and DLLs you also need to define ShareMM, ShareMMIfLibrary and AttemptToUseSharedMM using the FastMM4 Options Interface program and add FastMM4.pas to the top of the uses section of the .dpr for both the main application and the DLL and follow the directions outlined in item 3. This will allow FastMM to report memory leaks across EXEs and DLLs.
What’s next?

That’s it! Now you are ready to detect memory errors like never before. Fire up your IDE and start plugging those leaks!

1.6 Locking vs. Messaging

I believe that locking is evil. It leads to slow code and deadlocks and is one of the main reasons for almost-working multithreaded code (especially when you use shared data and forget to lock it up). Because of that, OmniThreadLibrary tries to move as much away from the shared data approach as possible. Cooperation between threads is rather achieved with messaging.

If we compare shared data approach with messaging, both have good and bad sides. On the good side, shared data approach is fast because it doesn’t move data around and is less memory intensive as the data is kept only in one copy. On the bad side, locking must be used to access data which leads to bad scaling (slowdowns when many threads are accessing the data), deadlocks and livelocks.

The situation is almost reversed for the messaging. There’s no shared data so no locking, which makes the program faster, more scalable and less prone to fall in the deadlocking trap. (Livelocking is still possible, though.) On the bad side, it uses more memory, requires copying data around (which may be a problem if shared data is large) and may lead to complicated and hard to understand algorithms.

OmniThreadLibrary uses custom lock-free structures to transfer data between the task and its owner (or directly between two tasks). The system is tuned for high data rates and can transfer more than million messages per second. However, in some situations shared data approach is necessary and that’s why OmniThreadLibrary adds significant support for synchronisation.

Some would disagree with the OmniThreadLibrary communication structures being called lock-free. In reality, there are no communication mechanisms that would correctly work in a multithreaded world without using locking. The name lock-free only implies that no operating system locking primitives are being used. Instead, OmniThreadLibrary achieves the thread-safeness by using bus locking, a special processor instruction prefix that achieves atomic operation in a multiprocessor system. Bus-locked operations are much slower than the normal assembler code, especially as they may stop other cores for the time of the operation execution, but then they are also much faster than the operating system locking.

Lock-free (or microlocked) structures in OmniThreadLibrary encompass:

• bounded (size-limited) stack
• bounded (size-limited) queue
• message queue
• dynamic (growing) queue
• blocking collection

OmniThreadLibrary automatically inserts two bounded queues between the task owner (IOmniTaskControl) and the task (IOmniTask) so that the messages can flow in both directions.

1.7 TOmniValue

A TOmniValue (part of the OtlCommon unit) is data type which is central to the whole OmniThreadLibrary. It is used in all parts of the code (for example in a communication subsystem) when type of the data that is to be stored/passed around is not known in advance.

It is implemeted as a smart record (a record with functions and operators) which functions similary to a Variant or TValue but is faster. It can store following data types:

• simple values (byte, integer, char, double, …)
• strings (Ansi, Unicode)
• Variant
• objects
• interfaces
• records (in D2009 and newer)

In all cases ownership of reference-counted data types (strings, interfaces) is managed correctly so no memory leaks can occur when such type is stored in a TOmniValue variable.

Two kinds of floating-point numbers can be stored in a TOmniValue – double and extended. Former are stored directly in the TOmniValue record while latter are wrapped in an TOmniExtendedData record, which increases memory usage and decreases performance. The use of double floating-point numbers is therefore recommended.

The TOmniValue type is too large to be shown in one piece so I’ll show various parts of its interface throughout this chapter.

1.7.1 Data Access

The content of a TOmniValue record can be accessed in many ways, the simplest (and in most cases the most useful) being through the AsXXX properties.

property AsAnsiString: AnsiString;
property AsBoolean: boolean;
property AsCardinal: cardinal;
property AsDouble: Double;
property AsDateTime: TDateTime;
property AsException: Exception;
property AsExtended: Extended;
property AsInt64: int64 read;
property AsInteger: integer;
property AsInterface: IInterface;
property AsObject: TObject;
property AsOwnedObject: TObject;
property AsPointer: pointer;
property AsString: string;
property AsVariant: Variant;
property AsWideString: WideString;

Exceptions can be stored through the AsObject property, but there’s also a special support for Exception data type with its own data access property AsException. It is extensively used in the Pipeline high-level abstraction.

While the setters for those properties are pretty straightforward, getters all have a special logic built in which tries to convert data from any reasonable source type to the requested type. If that cannot be done, an exception is raised.

For example, getter for the AsString property is called CastToString and in turn calls TryCastToString, which is in turn a public function of TOmniValue.

function TOmniValue.CastToString: string;
begin
if not TryCastToString(Result) then
raise Exception.Create('TOmniValue cannot be converted to string');
end;
function TOmniValue.TryCastToString(var value: string): boolean;
 begin
 Result := true;
   case ovType of
     ovtNull:       value := '';
     ovtBoolean:    value := BoolToStr(AsBoolean, true);
     ovtInteger:    value := IntToStr(ovData);
     ovtDouble,
     ovtDateTime,
     ovtExtended:   value := FloatToStr(AsExtended);
     ovtAnsiString: value := string((ovIntf as IOmniAnsiStringData).Value);
     ovtString:     value := (ovIntf as IOmniStringData).Value;
     ovtWideString: value := (ovIntf as IOmniWideStringData).Value;
     ovtVariant:    value := string(AsVariant);
     else Result := false;
   end;
 end;

When you don’t know the data type stored in a TOmniValue variable and you don’t want to raise an exception if compatible data is not available, you can use the TryCastToXXX family of functions directly.

function  TryCastToAnsiString(var value: AnsiString): boolean;
 function  TryCastToBoolean(var value: boolean): boolean;
 function  TryCastToCardinal(var value: cardinal): boolean;
 function  TryCastToDouble(var value: Double): boolean;
 function  TryCastToDateTime(var value: TDateTime): boolean;
 function  TryCastToException(var value: Exception): boolean;
 function  TryCastToExtended(var value: Extended): boolean;
 function  TryCastToInt64(var value: int64): boolean;
 function  TryCastToInteger(var value: integer): boolean;
 function  TryCastToInterface(var value: IInterface): boolean;
 function  TryCastToObject(var value: TObject): boolean;
 function  TryCastToPointer(var value: pointer): boolean;
 function  TryCastToString(var value: string): boolean;
 function  TryCastToVariant(var value: Variant): boolean;
 function  TryCastToWideString(var value: WideString): boolean;

Alternatively, you can use CastToXXXDef functions which return a default value if current value of the TOmniValue cannot be converted into required data type.

function  CastToAnsiStringDef(const defValue: AnsiString): AnsiString;
 function  CastToBooleanDef(defValue: boolean): boolean;
 function  CastToCardinalDef(defValue: cardinal): cardinal;
 function  CastToDoubleDef(defValue: Double): Double;
 function  CastToDateTimeDef(defValue: TDateTime): TDateTime;
 function  CastToExceptionDef(defValue: Exception): 
 Exception;
 function  CastToExtendedDef(defValue: Extended): Extended;
 function  CastToInt64Def(defValue: int64): int64;
 function  CastToIntegerDef(defValue: integer): integer;
 function  CastToInterfaceDef(const defValue: IInterface): IInterface;
 function  CastToObjectDef(defValue: TObject): TObject;
 function  CastToPointerDef(defValue: pointer): pointer;
 function  CastToStringDef(const defValue: string): string;
 function  CastToVariantDef(defValue: Variant): Variant;
 function  CastToWideStringDef(defValue: WideString): WideString;

They are all implemented in the same value, similar to the CastToObjectDef below.

function TOmniValue.CastToObjectDef(defValue: TObject): TObject;
 begin
   if not TryCastToObject(Result) then
     Result := defValue;
 end;

1.5 Tasks vs. Threads - OmniThreadLibrary

1.5 Tasks vs. Threads

In OmniThreadLibrary you don’t create threads but tasks. A task can be executed in a new thread or in an existing thread, taken from the thread pool.

A task is created using CreateTask function, which takes as a parameter a global procedure, a method, an instance of a TOmniWorker class (or, usually, a descendant of that class) or an anonymous method (in Delphi 2009 and newer). CreateTaskreturns an IOmniTaskControl interface, which can be used to control the task. A task is always created in suspended state and you have to call Run to activate it (or Schedule to run it in a thread pool).

The task has access to the IOmniTask interface and can use it to communicate with the owner (the part of the program that started the task). Both interfaces are explained in full detail in chapter Low-level multithreading.

The distinction between the task and the thread can be summarized in few simple words.

Task is part of code that has to be executed.

Thread is the execution environment.

You take care of the task, OmniThreadLibrary takes care of the thread.

Introduction to OmniThreadLibrary [2]

1.3 Installation

1. Download the last stable edition (download link is available at the OmniThreadLibrary site, or download the latest state from the repository. Typically, it is safe to follow the repository trunk as only tested code is committed. [Saying that, I have to admit that from time to time a bug or two do creep in but they are promptly exterminated].
2. If you have downloaded the last stable edition, unpack it to a folder.
3. Add the folder where you [unpacked last stable edition/checked out the SVN trunk] to the Delphi’s Library path. Also add the src subfolder to the Library path. In case you are already using units from my GpDelphiUnits
4. Add necessary units to the uses statement and start using the library! 

If you have XE8 or newer, you can download and install OmniThreadLibrary with the built-in GetIt! package manager. In that case you don’t have to manually add OmniThreadLibrary paths and you can also skip the next step (Installing Design Package). Both will be done automatically by the GetIt!

1.3.1 Installing Design Package

OmniThreadLibrary includes one design-time component (TOmniEventMonitor) which may be used to receive messages sent from the background tasks and to monitor thread creation/destruction. It is used in some of the demo applications.

To compile and install the package containing this component, follow these steps:

• From Delphi, open packages subfolder of the OmniThreadLibrary installation and select file OmniThreadLibraryPackages{VER}.groupproj (where {VER} indicates the version of the Delphi you’re using; at the moment of writing {VER} could be 2007, 2009, 2010, XE, XE2, XE3, XE4, XE5, XE6, XE7, or XE8).
• In the Project Manager window you’ll find two projects – OmniThreadLibraryRuntime{VER}.bpl and OmniThreadLibraryDesigntime{VER}.bpl. If the Project Manager window is not visible, select View, Project Manager from the menu.

• Right-click on the OmniThreadLibraryRuntime{VER}.bpland select Build from the pop-up menu.
• Right-click on the OmniThreadLibraryDesigntime{VER}.bpl and select Buildfrom the pop-up menu.
• Right-click again on the OmniThreadLibraryDesigntime{VER}.bpl and select Installfrom the pop-up menu.
• Delphi will report that the TOmniEventMonitor component was installed.

• Close the project group with File, Close All. If Delphi asks you whether to save modified files, choose No.

You should repeat these steps whenever the OmniThreadLibrary installation is updated.

1.4 Why Use OmniThreadLibrary?

OmniThreadLibrary approaches the threading problem from a different perspective than TThread. While the Delphi’s native approach is oriented towards creating and managing threads on a very low level, the main design guideline behind OmniThreadLibrary is: “Enable the programmer to work with threads in as fluent way as possible.” The code should ideally relieve you from all burdens commonly associated with multithreading.

OmniThreadLibrary was designed to become a “VCL for multithreading” – a library that will make typical multithreading tasks really simple but still allow you to dig deeper and mess with the multithreading code at the operating system level. While still allowing this low-level tinkering, OmniThreadLibrary allows you to work on a higher level of abstraction most of the time.

There are two important points of distinction between TThread and OmniThreadLibrary, both explained further in this chapter. One is that OmniThreadLibrary focuses on tasks, not threads and another is that in OmniThreadLibrary messaging tries to replace locking whenever possible.

By moving most of the critical multithreaded code into reusable components (classes and high-level abstractions), OmniThreadLibrary allows you to write better multithreaded code faster.

Introduction to OmniThreadLibrary

OmniThreadLibrary is a multithreading library for Delphi, written mostly by the author of this book (see Credits for full list of contributors). OmniThreadLibrary can be roughly divided into three parts. Firstly, there are building blocks that can be used either with the OmniThreadLibrary threading helpers or with any other threading approach (f.i. with Delphi’s TThread or with AsyncCalls). Most of these building blocks are described in chapter Miscellaneous, while some parts are covered elsewhere in the book (Lock-free Collections, Blocking collection, Synchronization).

Secondly, OmniThreadLibrary brings low-level multithreadingframework, which can be thought of as a scaffolding that wraps the TThread class. This framework simplifies passing messages to and from the background threads, starting background tasks, using thread pools and more.

Thirdly, OmniThreadLibrary introduces high-level multithreadingconcept. High-level framework contains multiple pre-packaged solutions (so-called abstractions; f.i. parallel for, pipeline, fork/join …) which can be used in your code. The idea is that the user should just choose appropriate abstraction and write the worker code, while the OmniThreadLibrary provides the framework that implements the tricky multithreaded parts, takes care of synchronisation and so on.

1.1 Requirements

OmniThreadLibrary requires at least Delphi 2007 and doesn’t work with FreePascal. The reason for this is that most parts of OmniThreadLibrary use language constructs that are not yet supported by the FreePascal compiler.

High-level multithreading framework requires at least Delphi 2009.

OmniThreadLibrary currently only targets Windows installation. Both 32-bit and 64-bit platform are supported.

1.2 License

OmniThreadLibrary is an open-sourced library with the OpenBSD license.

This software is distributed under the BSD license.

Copyright (c) 2015, Primoz Gabrijelcic

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• The name of the Primoz Gabrijelcic may not be used to endorse or promote products derived from this software without specific prior written permission.

In short, this means that:

1. You can use the library in any project, free, open source or commercial, without having to mention my name or the name of the library anywhere in your project, documentation or on the web site.
2. You can change the source for your own use. You can also put a modified version on the web, but you must not remove my name or the license from the source code.
3. I’m not guilty if the software blows in your face. Remember, you got OmniThreadLibrary for free.

3.4 Task Controller Needs an Owner

Task Controller Needs an Owner
The IOmniTaskController interface returned from the CreateTask must always be stored in a variable/field with a scope that exceeds the lifetime of the background task. In other words, don’t store a long-term background task interface in a local variable.

The simplest example of the wrong approach can be written in one line:

CreateTask(MyWorker).Run;

This code looks fine, but it doesn’t work. In this case, the IOmniTaskController interface is stored in a hidden temporary variable which is destroyed at the end of the current method. This then causes the task controller to be destroyed which in turn causes the background task to be destroyed. Running this code would therefore just create and then destroy the task.

A common solution is to just store the interface in some field.

FTaskControl := CreateTask(MyWorker).Run;

When you don’t need background worker anymore, you should terminate the task and free the task controller.

FTaskControl.Terminate;
FTaskControl := nil;

Another solution is to provide the task with an implicit owner. You can, for example, use the event monitor to monitor tasks lifetime or messages sent from the task and that will make the task owned by the monitor. The following code is therefore valid:

CreateTask(MyWorker).MonitorWith(eventMonitor).Run;

Yet another possibility is to call the Unobserved) before the Run. This method makes the task being observed by an internal monitor.

CreateTask(MyWorker).Unobserved.Run;

When you use a thread pool to run a task, the thread pool acts as a task owner so there’s no need for an additional explicit owner.

procedure Beep(const task: IOmniTask);
begin
   MessageBeep(MB_ICONEXCLAMATION);
 end;
 
 CreateTask(Beep, 'Beep').Schedule;