Skip to content
rob mathews edited this page Jun 22, 2021 · 9 revisions

tl;dr

EDI sucks and all the libraries suck; It's a vortex of suckage.

You want this version if you work for a living and have production sized files. If you have smaller transaction sizes < 200 records, I would recommend stupidedi, it's much nicer and well designed.

history

This gem is a version of the many X12 parsing and generation gems that were forked from an original perl implementation about a decade or so ago. Those interested in reading the complete provenance of this gem can find it somewhere in here.

why this version?

This version differs from all of those fairly useless ports in that it can actually handle large files - I'm using it to handle EDI 834 file of ~9000 eligibility records.

For the record, I tried using stupidedi first. It was performant on ~300 records, but couldn't parse 9000 records - it consumed more than 20GB and 24 hours before I had to kill the process, never finishing.

As you can tell from my fairly messy commit history, three major improvements were made.

  • Use a string compare before invoking a regex match. Regex compares that fail were surprisingly expensive.
  • convert tail-recursion to iteration, otherwise you run out of stack after about 4000 records
  • add an option to process a completed record via a callback, which if you do that the gem can throw out the record and reclaim the memory. (Otherwise, the gem builds up about 12GB of memory on 9000 records).

You can read the usage over here, it's mostly all still valid. I'll just document the new callback feature here, otherwise it is all the same.

[Callbacks]

Clone this wiki locally