Architecture

The Q system is a distributed system which consists of cooperating applications. All these applications have a common denominator: they react to input events by producing output as fast as possible. The applications within the Q system are event-driven.

The typical design of event-driven applications is based on a handling-loop around a queue of events. These events represent signals from inside and outside the application that trigger the application to react. Note that a reaction could also be generating more signals for other applications to react to. Examples of such signals can be:

• • Data arriving on a network card.

  • Operating system signals.

  • Application events, either local, in other threads, or other applications.

High frequency trading applications can be regarded as high performance event-driven applications. Received market data trigger applications to enter, replace, or cancel orders as fast as possible. Because these applications are vital to a trading organisation, the applications must comply to very stringent quality constraints while providing their functionality:

• Performance

  • • Low-latency, react as fast as possible on incoming market data.

    • High-throughput, process a lot of market data as fast as possible.

    • Scalable, split up CPU-consuming functionalities and distribute them over other CPUs and/or other machines.

• Modifiability

  • • High degree of extendability, add new functionalities, e.g. access to new exchanges, new trading strategies, new applications.

    • High degree of maintainability, adapt and maintain existing functionalities, e.g. exchange migrations, bug fixing.

• • Availability, run 24x7.

  • Interoperability, high degree of cooperation between applications.

In order to comply to these constraints, all applications within the Q system are based on the same event-driven architecture. This event-driven architecture is like a box of lego containing reusable fundamental building blocks to create event-driven applications. These building blocks provide key solutions and core mechanisms while complying to the necessary stringent quality constraints at the same time.

The architecture provides building blocks for the following key solutions and core mechanisms:

• • event-handling, see also the reactor pattern

  • coding and decoding, binary, ASCII, FIX, FAST

  • i/o session management

    • non-blocking i/o

    • zero-copy i/o

    • recvmmsg

  • ready framework, central point to determine whether a group of i/o sessions is ready for trade or not.

  • threading

  • assigning threads to CPUs, analyse CPU cache layouts, see also hwloc

  • lock-free concurrent algorithms for queues and sets

  • low-latency logging

  • code generation

To demonstrate the performance capabilities of the architecture, a benchmark is available.