Home

tl;dr

EDI sucks and all the libraries suck; It's a vortex of suckage.

You want this version if you work for a living and have production sized files. If you have smaller transaction sizes < 200 records, I would recommend stupidedi, it's much nicer and well designed.

history

This gem is a version of the many X12 parsing and generation gems that were forked from an original perl implementation about a decade or so ago. Those interested in reading the complete provenance of this gem can find it somewhere in here.

why this version?

This version differs from all of those fairly useless ports in that it can actually handle large files - I'm using it to handle an EDI 834 file of ~310K eligibility records.

For the record, I tried using stupidedi first. It was performant on ~300 records, but couldn't parse 300K records - it consumed more than 20GB and 24 hours before I had to kill the process, never finishing.

As you can tell from my fairly messy commit history, three major improvements were made.

• Use a string compare before invoking a regex match. Regex compares that fail were surprisingly expensive.
• pays attention to the "max=nnn" parameter and doesn't attempt to parse more after nnn records are found.
• convert tail-recursion to iteration, otherwise you run out of stack after about 4000 records
• add an option to process a completed record via a callback, which if you do that the gem can throw out the record and reclaim the memory. (Otherwise, the gem builds up about 12GB of memory before process more than 4000 records, after which I killed it).

Also, since I was parsing 834 files, I fixed the 834.xml file to be able to actually parse. The one that this gem was based on had multiple errors. FWIW, I removed the L2100C, L2100D definitions, since my clients 834 file didn't have them, and removing them gave a 25% performance improvement. You might need to add them back.

You can read the usage over here, it's mostly all still valid. I'll just document the new callback feature here, otherwise it is all the same.