ARP cache failed

[AmyNova]

Nova user received the following error:

Expiring OS fingerprint for xxx.xxx.xx.xx
honeyd: config.c:721: template_remove_arp: Assertion 'arp ! = ((void*)0)' failed.
Failed to start Haystack

It seems like the honeyd ARP table is being asked for a value that it does not have – so it fails to start the haystack. Honeyd’s ARP mechanisms were never developed by the honeyd project to mirror the ARP caching on Linux. We have identified some differences in this on one of the issues in the Github issue list for honeyd: (see issue #44)

#44

If the user can re-start honeyd and it does not happen again then we can assume it is a caching behavior. If the error can be re-produced, it becomes a issue that we need to investigate further. Please comment if others have seen this issue.

[alphaDev23]

This appears to be the same issue:

DHCP lease renewed with new lease of 43200 seconds
honeyd: config.c:721: template_remove_arp: Assertion 'arp ! = ((void*)0)' failed.
Failed to start Haystack

It is running on Ubuntu 12.04.4. Please let me know if this can be resolved.

[DarkmatterVale]

dwebber, I would reinstall Honeyd.

Did that fix the issue?

[alphaDev23]

I had just reinstalled, so no that does not fix the issue.

Same error again today. The 1st line above regarding dhcp renewal appears to have nothing to do with the error.

Please advise.

From: DarkmatterVale [mailto:[email protected]]
Sent: Friday, July 11, 2014 05:38
To: DataSoft/Honeyd
Cc: dwebber
Subject: Re: [Honeyd] ARP cache failed (#87)

dwebber, I would reinstall Honeyd.

Did that fix the issue?

—
Reply to this email directly or view it on GitHub #87 (comment) . https://github.com/notifications/beacon/250169__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDY5Nzg2MCwiZGF0YSI6eyJpZCI6MjAwNDUwOTl9fQ==--c8c1d4bbcca9111dd3015f3d2220b56dafef090f.gif

[awaldow]

I would check that you don't have duplicate MACs for nodes, and I'd also try and clear your/your router's ARP cache to see if that fixes the issue.

[alphaDev23]

I have another installation running fine using only 1 listening interface (settings -> Ethernet Interface). On the install in question, I’m running on multiple interfaces but the UI does not allow to select only 2 out of the 4 interfaces on the system. The UI either forces only 1 or all 4. Issue is noticed when running on all 4 interfaces. Is there a way to manually set the listing interfaces to only 2 or 3 of the 4?

Resolving this will at least allow me to determine if it is interface specific.

From: Addison Waldow [mailto:[email protected]]
Sent: Friday, July 11, 2014 08:17
To: DataSoft/Honeyd
Cc: dwebber
Subject: Re: [Honeyd] ARP cache failed (#87)

I would check that you don't have duplicate MACs for nodes, and I'd also try and clear your/your router's ARP cache to see if that fixes the issue.

—
Reply to this email directly or view it on GitHub #87 (comment) . https://github.com/notifications/beacon/250169__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDcwNzQyMCwiZGF0YSI6eyJpZCI6MjAwNDUwOTl9fQ==--806c0d0e1efeb408db4af74f05e0b2bb5bfdd93a.gif

[awaldow]

I would think it would be easier to determine if there's an interface specific issue by doing them one at a time. But that won't really help, as you have discovered a bug wherein only the last interface of the selected ones is actually saved. I will open a new ticket for this.

[alphaDev23]

While it is not saved via the interface, can this be set in a config file somewhere? If yes, where is the config file?

From: Addison Waldow [mailto:[email protected]]
Sent: Friday, July 11, 2014 10:29
To: DataSoft/Honeyd
Cc: dwebber
Subject: Re: [Honeyd] ARP cache failed (#87)

I would think it would be easier to determine if there's an interface specific issue by doing them one at a time. But that won't really help, as you have discovered a bug wherein only the last interface of the selected ones is actually saved. I will open a new ticket for this.

—
Reply to this email directly or view it on GitHub #87 (comment) . https://github.com/notifications/beacon/250169__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDcxNTM0OSwiZGF0YSI6eyJpZCI6MjAwNDUwOTl9fQ==--cf3eac5273685b0802f28f2a6597c6933dcd7cea.gif

[awaldow]

I believe that if you change the value of the INTERFACE variable in the NOVAConfig.txt file this will satisfy your needs. This value is loaded on init, so you'll have to restart. But you should get by with a space delimited list (i.e. INTERFACES eth0 eth2 eth3)

[DarkmatterVale]

Can you select individual interfaces in the node that works? This would tell you if it is just a single node issue, or overall issue.

[alphaDev23]

Yes, but I already have a different install running on a single interface without issue. I believe the issue may occur with multiple interfaces.

I changed the INTERFACE variable on NOVAConfig.txt to listen to more than one interface and I’m testing now. If it runs through the weekend we should be good.

Note, even though it is listening to 2 interfaces, per the IPs specified on the Nodes tab, Settings only checks the 1st interface specified in the NOVAConfig.txt file.

From: DarkmatterVale [mailto:[email protected]]
Sent: Friday, July 11, 2014 11:23
To: DataSoft/Honeyd
Cc: dwebber
Subject: Re: [Honeyd] ARP cache failed (#87)

Can you select individual interfaces in the node that works?

—
Reply to this email directly or view it on GitHub #87 (comment) . https://github.com/notifications/beacon/250169__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDcxODU4MywiZGF0YSI6eyJpZCI6MjAwNDUwOTl9fQ==--aed262ad89b7b9628413f07f6ab959901b509095.gif

[DarkmatterVale]

awaldow, is their any way to force the starting of the interfaces? One that is not permanent, but works only in that instance, or session

[awaldow]

What do you mean DV?

[DarkmatterVale]

Is there any way to start the interfaces without going through the UI? Because then we could try to start just the 1st and the 2nd interfaces, and see if the issue still occurs. Do you see what I am saying?

This would just help narrow down the problem

[alphaDev23]

Just as a note, the interface start per the NOVAConfig.txt file. It is just not reflected accurately in the UI.

From: Addison Waldow [mailto:[email protected]]
Sent: Friday, July 11, 2014 12:55
To: DataSoft/Honeyd
Cc: dwebber
Subject: Re: [Honeyd] ARP cache failed (#87)

What do you mean DV?

—
Reply to this email directly or view it on GitHub #87 (comment) . https://github.com/notifications/beacon/250169__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDcyNDEwNiwiZGF0YSI6eyJpZCI6MjAwNDUwOTl9fQ==--7b1e2d23758c6adb98df18ebcaf5cbdc0cb51423.gif

[DarkmatterVale]

Ok, thanks. Sorry, but I am not as technically adept as awaldow is in this.

[alphaDev23]

Is there anyway to put logging around this error?

honeyd: config.c:721: template_remove_arp: Assertion 'arp ! = ((void*)0)' failed.
Failed to start Haystack

This is still occurring when running on multiple interfaces. Could there be an issue with separate DHCP servers, one for each interface?

[alphaDev23]

Please advise.

[awaldow]

It's an assertion so it's supposed to error out when that expression fails. Which means there is some issue with your setup regarding ARP. Could be the same MAC on two different interfaces, could be some connection/router issue with its ARP table. I

[DarkmatterVale]

Have you tried running your device at a different location?

[alphaDev23]

Is there a log or way to further log this issue? The information provided does not give a lot of information. For example, if it is a MAC issue, knowing which MAC address is causing the issue would help.

I’ve run this on a on another machine with the same results.