Comments for 29West http://blog.29west.com High-Performance Messaging Wed, 07 Oct 2009 22:10:54 +0000 http://wordpress.com/ hourly 1 Comment on Estimating Errors in Round-Trip Latency Measurements due to Clock Drift by Estimating Errors in Round-Trip Latency Measurements due to Clock Drift « Rajan’s Blog http://blog.29west.com/2009/09/01/errors-round-trip-latency-measurements-clock-drift/#comment-20 Estimating Errors in Round-Trip Latency Measurements due to Clock Drift « Rajan’s Blog Wed, 07 Oct 2009 22:10:54 +0000 http://blog.29west.com/?p=574#comment-20 [...] Estimating Errors in Round-Trip Latency Measurements due to Clock Drift [...] [...] Estimating Errors in Round-Trip Latency Measurements due to Clock Drift [...]

]]> Comment on Do you want your exchange feed normalized? by Neil Horlock http://blog.29west.com/2009/01/28/do-you-want-your-exchange-feed-normalized/#comment-5 Neil Horlock Wed, 04 Feb 2009 14:25:59 +0000 http://29west.wordpress.com/?p=72#comment-5 Normalization comes in at more than one level. 1) Content normalization: The most destructive form where data is remapped to a homogenous presentation, and a defined set of fields published in a defined format. e.g.An automated multi-market smart order routing system may need to know the trading state of a product at a given venue, for the most part it needs to know only the state at an abstract level (is it tradable or not) while the venues will all have nuances and at the very least different presentations. 2) Transport normalization: Data is not remapped, nor discarded but is repackaged in a common protocol, the line between type 1 and type 2 blurs of course but often even applications that would turn their noses up at the "de-valued" data in option 1 would still rather receive the data in a common "package". Quite what transformations occur is very much use case dependent but an example would be the conversion of binary prices into ASCII or vice versa, or publication over a common market data back bone. In a world where compression such as FAST is becoming more normal number 2 above is almost a given. Consuming applications are very unlikely to want to deal with the specific decompression rules and thus the data is going to be extracted and in the process you have made a normalisation decision. Normalization comes in at more than one level.
1) Content normalization: The most destructive form where data is remapped to a homogenous presentation, and a defined set of fields published in a defined format. e.g.An automated multi-market smart order routing system may need to know the trading state of a product at a given venue, for the most part it needs to know only the state at an abstract level (is it tradable or not) while the venues will all have nuances and at the very least different presentations.

2) Transport normalization: Data is not remapped, nor discarded but is repackaged in a common protocol, the line between type 1 and type 2 blurs of course but often even applications that would turn their noses up at the “de-valued” data in option 1 would still rather receive the data in a common “package”. Quite what transformations occur is very much use case dependent but an example would be the conversion of binary prices into ASCII or vice versa, or publication over a common market data back bone.

In a world where compression such as FAST is becoming more normal number 2 above is almost a given. Consuming applications are very unlikely to want to deal with the specific decompression rules and thus the data is going to be extracted and in the process you have made a normalisation decision.

]]> Comment on Do you want your exchange feed normalized? by David Walthour http://blog.29west.com/2009/01/28/do-you-want-your-exchange-feed-normalized/#comment-4 David Walthour Fri, 30 Jan 2009 03:01:02 +0000 http://29west.wordpress.com/?p=72#comment-4 I agree that it is largely application dependent. Some applications need to deal with the nuances of the direct exchange data while many don't. Normalizing can greatly simplify the design considerations for many apps. Also, if the normalization is combined with throttling, the result can be much better overall network throughput and performance for applications that are not latency sensitive, e.g., non trading and execution apps. The majority of applications that I write using LBM feed handlers would greatly benefit from normalized exchange feeds. I agree that it is largely application dependent. Some applications need to deal with the nuances of the direct exchange data while many don’t. Normalizing can greatly simplify the design considerations for many apps. Also, if the normalization is combined with throttling, the result can be much better overall network throughput and performance for applications that are not latency sensitive, e.g., non trading and execution apps. The majority of applications that I write using LBM feed handlers would greatly benefit from normalized exchange feeds.

]]> Comment on Do you want your exchange feed normalized? by Jerry Nelligan http://blog.29west.com/2009/01/28/do-you-want-your-exchange-feed-normalized/#comment-3 Jerry Nelligan Thu, 29 Jan 2009 22:14:52 +0000 http://29west.wordpress.com/?p=72#comment-3 It depends on my application. In applications that are latency and resource intensive, I do not want my feed normalized. Normalizing the feed takes precious CPU resources at the point of contact. It can also potentially delete or change some of the subtle data provided by the exchange. When someone normalizes the data for me, I may not be aware of changes to that data and their impact on my business since I am not involved in the normalization process. For applications that don't have extreme latency sensitivity, such as user front ends or risk applications, I would like my data normalized as much as possible. It depends on my application. In applications that are latency and resource intensive, I do not want my feed normalized. Normalizing the feed takes precious CPU resources at the point of contact. It can also potentially delete or change some of the subtle data provided by the exchange. When someone normalizes the data for me, I may not be aware of changes to that data and their impact on my business since I am not involved in the normalization process.

For applications that don’t have extreme latency sensitivity, such as user front ends or risk applications, I would like my data normalized as much as possible.

]]>