The feature of persistence plugins

[p-alik]

... The whole point of gearmand is ruined by the queue plugins. It's supposed to be a many-to-many scale out stateless daemon. I figure most of the confusion happened when libgearman was written wrong, with no hash-unique-to-select-server logic, and so people lost the real purpose of the protocol and the daemon. You're supposed to run n+1 gearmands, client side load balance between them (by hashing the unique against the list of servers, just like memcached, which was designed by the same team), and never mind if you lose a gearmand.

... I know some people have found their happy place with the queue plugins. I won't ever block their continued investment and existence. But I will remind everyone I can that you are absolutely doing it wrong...

@SpamapS - Being a gearmand user since the original Danga days, I strongly second your sentiment.

I don't know the full history of the plugins, but obviously the "do one thing and do it well" credo was lost somewhere along the way. The most egregious deviation was, IIRC, the "memcached persistence plugin" -- misnomer to say the least -- but also indicative of the loss of clarity and the deviated path libgearman took.

Ideally, this project would iterate on abiding by the gearmand protocol (fixing bugs, improving performance, etc), and nothing more.

There are a few core issues that fall under this purview and should warrant highest priority:

• libgearman isn't hashing submitted jobs based on unique (I believe this is what @SpamapS was alluding to above)
• Unexpected Worker Timeout (CAN_DO_TIMEOUT) Behavior
• ... others?

"Plugins" would exist separately, elsewhere, for those that desire them. Curious, @p-alik, does Perl gearmand have "plugin architecture" akin to what's seen here?

Joe Stump, back in the famed Digg days, said IIRC, on the Mailing List something like, "[Gearman] is a strange and wonderful beast. It takes a while to see it's true brilliance."

Gearman is, and has always been, truly brilliant. These plugins have obfuscated and undermined that fact to more general audiences.

Originally posted by @bmeynell in #216 (comment)

[bmeynell]

@p-alik - Would it be beneficial to apply a core or plugin label to every issue?

I inventoried the 65 current open issues and this is what I came up with:

plugin: #6, #10, #28, #29, #45, #56, #59, #61, #63, #69, #73, #94, #98, #99, #112, #113, #117, #119, #126, #127, #136, #156, #157, #159, #207, #224, #227

core: #7, #20, #24, #35, #37, #41, #42, #48, #50, #51, #54, #58, #64, #67, #68, #81, #101, #102, #110, #116, #125, #128, #134, #138, #141, #148, #150, #158, #160, #163, #166, #196, #197, #198, #199, #200, #222, #226

I did not inventory the Gearman Google Group or the old Launchpad Bug Tracker (each for issues, ideas, bugs, etc., reported those two places, but not transferred here).

[SpamapS]

Helping us categorize things is a great idea, definitely helps folks who want to take ownership of some but not all.

I'd really love to address the non-hashing of the keys issue, but every time I sit down to do it, I'm frankly disgusted by C/C++ and run back to learning Rust so we can do this right. My ideal world is one where libgearman is just a C/C++ binding around a Rust implementation. Unfortunately, I've very little time to play with that, so if somebody wants to just fix libgearman I'd be forever grateful. :)