Up to date to GeoEvent 10.6.1 on June 27, 2022.
Ingesting Variable Size CSV Messages
In GeoEvent, you should use a Textual content Adapter on an inbound connector to parse comma separated values (CSV), usually with both the TCP transport (like once you use the GeoEvent Simulator) or the File transport (to observe a folder for information recordsdata). Usually, these CSV messages are typically static in format permitting you to make use of one GeoEvent definition to learn them in. Nevertheless, I not too long ago ran right into a scenario the place a tool was reporting a truncated message, primarily based on the quantity of information it really needed to report. On this case, the gadgets had been solely reporting updates, so if there was no replace for fields on the finish of the message, these fields had been omitted from the CSV message. A pattern of the info would seem like this (I’ve added some areas to make the construction a bit clearer):
Title,IdNum,Sort,MsgDatetime ,Value1,Value2,Value3,Value4,Value5,Value6,Value7,Value8,Value9,Value10
Red1,1 ,Purple ,1585947092133, , , , , ,20.402, , , ,-0.57
Red1,1 ,Purple ,1585947692133, , , , , ,1.5001, ,148 ,2
Red1,1 ,Purple ,1585948292133, ,10 ,62 ,4 ,-0.22 ,17.701,170 , ,1
Red1,1 ,Purple ,1585948892133,6 , ,65 , ,0.64 , ,220 , ,8
Red1,1 ,Purple ,1585949492133, , , ,1
Red1,1 ,Purple ,1585950092133,94 ,15 , ,1 , , ,480 ,139 ,3
Red1,1 ,Purple ,1585950692133,42 ,3 , , ,0.64 , , , ,4 ,-0.99
Red1,1 ,Purple ,1585951292133,67 ,-17 , , ,-0.05 ,5.4005, , , ,-0.74
Red1,1 ,Purple ,1585951892133, , ,15 ,3 ,-0.72 , ,280 , ,7
Red1,1 ,Purple ,1585952492133,22 ,14 , , ,-0.33 ,28.502, , ,8
Red1,1 ,Purple ,1585953092133, ,2 , ,1 , , ,395 , ,3
Red1,1 ,Purple ,1585953692133, , , ,1 , , , , ,4
Red1,1 ,Purple ,1585954292133,55 ,3 ,20 , , , , , ,8
Red1,1 ,Purple ,1585954892133,97 ,8 , , ,-0.2 , ,470 , ,8 ,-0.71
Utilizing the OOTB CSV inputs, all however 4 of those messages could be dropped as a result of they do not include the complete set of fields. All the messages conform to the anticipated GeoEvent Definition, nevertheless a few of them are lacking the trailing set of fields as a result of there isn’t any up to date worth for these fields.
Versatile CSV Adapter
To get round this challenge I up to date the OOTB inbound Textual content Adapter to be a bit extra versatile. For every CSV message, it can try to put every discipline into the offered GeoEvent definition. If there aren’t sufficient fields within the CSV message to fill the GeoEvent, the remaining fields within the occasion can have their values set to NULL. If there are too many fields within the CSV message, the additional fields within the CSV message will likely be ignored (like should you solely wished the primary 5 values within the above instance messages).
Deploying & Compatibility
This adapter is suitable with any model of GeoEvent Server 10.4 or later. To deploy, unzip the hooked up file and both add the .jar file to GeoEvent Supervisor on Web site > Elements > Adapters utilizing the Add Native Adapter button. Alternatively, you possibly can copy the .jar file into the deploy folder of your GeoEvent Server set up (for home windows that will usually be C:Program FilesArcGISServerGeoEventdeploy ). The present launch is marked model 0.1 (proof of idea). Please report any points within the feedback part of this weblog.
Connectors
Including the .jar file to GeoEvent Server will create two inbound connectors for you. You may create extra connectors using completely different transports if wanted.
Troubleshooting
Loggers that may assist troubleshoot:
com.esri.geoevent.adapter.flextext.TextInboundAdapterService
com.esri.geoevent.adapter.flextext.TextInboundAdapter
Supply Code
The supply code for this adapter is accessible within the following GitHub repository:
