1.6 Locking vs. Messaging

I believe that locking is evil. It leads to slow code and deadlocks and is one of the main reasons for almost-working multithreaded code (especially when you use shared data and forget to lock it up). Because of that, OmniThreadLibrary tries to move as much away from the shared data approach as possible. Cooperation between threads is rather achieved with messaging.

If we compare shared data approach with messaging, both have good and bad sides. On the good side, shared data approach is fast because it doesn’t move data around and is less memory intensive as the data is kept only in one copy. On the bad side, locking must be used to access data which leads to bad scaling (slowdowns when many threads are accessing the data), deadlocks and livelocks.

The situation is almost reversed for the messaging. There’s no shared data so no locking, which makes the program faster, more scalable and less prone to fall in the deadlocking trap. (Livelocking is still possible, though.) On the bad side, it uses more memory, requires copying data around (which may be a problem if shared data is large) and may lead to complicated and hard to understand algorithms.

OmniThreadLibrary uses custom lock-free structures to transfer data between the task and its owner (or directly between two tasks). The system is tuned for high data rates and can transfer more than million messages per second. However, in some situations shared data approach is necessary and that’s why OmniThreadLibrary adds significant support for synchronisation.

Some would disagree with the OmniThreadLibrary communication structures being called lock-free. In reality, there are no communication mechanisms that would correctly work in a multithreaded world without using locking. The name lock-free only implies that no operating system locking primitives are being used. Instead, OmniThreadLibrary achieves the thread-safeness by using bus locking, a special processor instruction prefix that achieves atomic operation in a multiprocessor system. Bus-locked operations are much slower than the normal assembler code, especially as they may stop other cores for the time of the operation execution, but then they are also much faster than the operating system locking.

Lock-free (or microlocked) structures in OmniThreadLibrary encompass:

• bounded (size-limited) stack
• bounded (size-limited) queue
• message queue
• dynamic (growing) queue
• blocking collection

OmniThreadLibrary automatically inserts two bounded queues between the task owner (IOmniTaskControl) and the task (IOmniTask) so that the messages can flow in both directions.

1.7 TOmniValue

A TOmniValue (part of the OtlCommon unit) is data type which is central to the whole OmniThreadLibrary. It is used in all parts of the code (for example in a communication subsystem) when type of the data that is to be stored/passed around is not known in advance.

It is implemeted as a smart record (a record with functions and operators) which functions similary to a Variant or TValue but is faster. It can store following data types:

• simple values (byte, integer, char, double, …)
• strings (Ansi, Unicode)
• Variant
• objects
• interfaces
• records (in D2009 and newer)

In all cases ownership of reference-counted data types (strings, interfaces) is managed correctly so no memory leaks can occur when such type is stored in a TOmniValue variable.

Two kinds of floating-point numbers can be stored in a TOmniValue – double and extended. Former are stored directly in the TOmniValue record while latter are wrapped in an TOmniExtendedData record, which increases memory usage and decreases performance. The use of double floating-point numbers is therefore recommended.

The TOmniValue type is too large to be shown in one piece so I’ll show various parts of its interface throughout this chapter.

1.7.1 Data Access

The content of a TOmniValue record can be accessed in many ways, the simplest (and in most cases the most useful) being through the AsXXX properties.

property AsAnsiString: AnsiString;
property AsBoolean: boolean;
property AsCardinal: cardinal;
property AsDouble: Double;
property AsDateTime: TDateTime;
property AsException: Exception;
property AsExtended: Extended;
property AsInt64: int64 read;
property AsInteger: integer;
property AsInterface: IInterface;
property AsObject: TObject;
property AsOwnedObject: TObject;
property AsPointer: pointer;
property AsString: string;
property AsVariant: Variant;
property AsWideString: WideString;

Exceptions can be stored through the AsObject property, but there’s also a special support for Exception data type with its own data access property AsException. It is extensively used in the Pipeline high-level abstraction.

While the setters for those properties are pretty straightforward, getters all have a special logic built in which tries to convert data from any reasonable source type to the requested type. If that cannot be done, an exception is raised.

For example, getter for the AsString property is called CastToString and in turn calls TryCastToString, which is in turn a public function of TOmniValue.

function TOmniValue.CastToString: string;
begin
if not TryCastToString(Result) then
raise Exception.Create('TOmniValue cannot be converted to string');
end;
function TOmniValue.TryCastToString(var value: string): boolean;
 begin
 Result := true;
   case ovType of
     ovtNull:       value := '';
     ovtBoolean:    value := BoolToStr(AsBoolean, true);
     ovtInteger:    value := IntToStr(ovData);
     ovtDouble,
     ovtDateTime,
     ovtExtended:   value := FloatToStr(AsExtended);
     ovtAnsiString: value := string((ovIntf as IOmniAnsiStringData).Value);
     ovtString:     value := (ovIntf as IOmniStringData).Value;
     ovtWideString: value := (ovIntf as IOmniWideStringData).Value;
     ovtVariant:    value := string(AsVariant);
     else Result := false;
   end;
 end;

When you don’t know the data type stored in a TOmniValue variable and you don’t want to raise an exception if compatible data is not available, you can use the TryCastToXXX family of functions directly.

function  TryCastToAnsiString(var value: AnsiString): boolean;
 function  TryCastToBoolean(var value: boolean): boolean;
 function  TryCastToCardinal(var value: cardinal): boolean;
 function  TryCastToDouble(var value: Double): boolean;
 function  TryCastToDateTime(var value: TDateTime): boolean;
 function  TryCastToException(var value: Exception): boolean;
 function  TryCastToExtended(var value: Extended): boolean;
 function  TryCastToInt64(var value: int64): boolean;
 function  TryCastToInteger(var value: integer): boolean;
 function  TryCastToInterface(var value: IInterface): boolean;
 function  TryCastToObject(var value: TObject): boolean;
 function  TryCastToPointer(var value: pointer): boolean;
 function  TryCastToString(var value: string): boolean;
 function  TryCastToVariant(var value: Variant): boolean;
 function  TryCastToWideString(var value: WideString): boolean;

Alternatively, you can use CastToXXXDef functions which return a default value if current value of the TOmniValue cannot be converted into required data type.

function  CastToAnsiStringDef(const defValue: AnsiString): AnsiString;
 function  CastToBooleanDef(defValue: boolean): boolean;
 function  CastToCardinalDef(defValue: cardinal): cardinal;
 function  CastToDoubleDef(defValue: Double): Double;
 function  CastToDateTimeDef(defValue: TDateTime): TDateTime;
 function  CastToExceptionDef(defValue: Exception): 
 Exception;
 function  CastToExtendedDef(defValue: Extended): Extended;
 function  CastToInt64Def(defValue: int64): int64;
 function  CastToIntegerDef(defValue: integer): integer;
 function  CastToInterfaceDef(const defValue: IInterface): IInterface;
 function  CastToObjectDef(defValue: TObject): TObject;
 function  CastToPointerDef(defValue: pointer): pointer;
 function  CastToStringDef(const defValue: string): string;
 function  CastToVariantDef(defValue: Variant): Variant;
 function  CastToWideStringDef(defValue: WideString): WideString;

They are all implemented in the same value, similar to the CastToObjectDef below.

function TOmniValue.CastToObjectDef(defValue: TObject): TObject;
 begin
   if not TryCastToObject(Result) then
     Result := defValue;
 end;