Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration tests fail to run in CI` #3

Open
ozten opened this issue Oct 25, 2011 · 5 comments
Open

Integration tests fail to run in CI` #3

ozten opened this issue Oct 25, 2011 · 5 comments

Comments

@ozten
Copy link
Owner

ozten commented Oct 25, 2011

https://jenkins.mozilla.org/job/sasl-browserid/

We now have unit tests running, but integration tests fail.

Repro Steps:
ssh into slave01
edit /home/hudson-slave/workspace/sasl-browserid/test/run.sh and un-comment functional_test.py
sudo /usr/sbin/chroot /home/hudson-slave/workspace/sasl-browserid /test/run.sh

Output:
sudo -u hudson-slave sudo /usr/sbin/chroot /home/hudson-slave/workspace/sasl-browserid /test/run.sh

Message from syslogd@dm-hudson01-slave01 at Oct 25 16:10:40 ...
browserid-client: browserid_client_plug_init_plugin initialized

.

Ran 1 test in 0.445s

OK
Resetting slapd
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
EResetting slapd
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
ldap_bind: Can't contact LDAP server (-1)
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
EResetting slapd
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
adding new entry "dc=example,dc=com"
ldap_result: Can't contact LDAP server (-1)

connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
EResetting slapd
adding new entry "dc=example,dc=com"
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
connecting ldap://127.0.0.1:1389/
still starting up
ldap_result: Can't contact LDAP server (-1)

connecting ldap://127.0.0.1:1389/
EResetting slapd
adding new entry "dc=example,dc=com"
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
ldap_result: Can't contact LDAP server (-1)

connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still loading
still loading
still loading
still loading
still loading
adding new entry "dc=example,dc=com"

adding new entry "uid=[email protected],dc=example,dc=com"
EResetting slapd
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
ldap_result: Can't contact LDAP server (-1)

connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still loading
still loading
still loading
adding new entry "dc=example,dc=com"
still loading

adding new entry "uid=[email protected],dc=example,dc=com"
EResetting slapd
killed
starting
slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
ldap_result: Can't contact LDAP server (-1)

connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
adding new entry "dc=example,dc=com"

adding new entry "uid=[email protected],dc=example,dc=com"

slapd -h ldap://127.0.0.1:1389/ -f /test/slapd/slapd.conf
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/
still starting up
connecting ldap://127.0.0.1:1389/

E

ERROR: test_bad_assertion_buffer_overflow_assertion (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 127, in test_bad_assertion_buffer_overflow_assertion
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/unittest.py", line 320, in failUnlessRaises
callableObj(_args, *_kwargs)
File "/test/functional_test.py", line 127, in
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_bad_assertion_buffer_overflow_audience (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 144, in test_bad_assertion_buffer_overflow_audience
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/unittest.py", line 320, in failUnlessRaises
callableObj(_args, *_kwargs)
File "/test/functional_test.py", line 144, in
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_bad_assertion_empty_string (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 119, in test_bad_assertion_empty_string
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/unittest.py", line 320, in failUnlessRaises
callableObj(_args, *_kwargs)
File "/test/functional_test.py", line 119, in
self.assertRaises(ldap.LOCAL_ERROR, lambda:
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_bad_assertion_no_auth (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 111, in test_bad_assertion_no_auth
self.assertRaises(ldap.INVALID_CREDENTIALS, lambda:
File "/usr/lib/python2.4/unittest.py", line 320, in failUnlessRaises
callableObj(_args, *_kwargs)
File "/test/functional_test.py", line 111, in
self.assertRaises(ldap.INVALID_CREDENTIALS, lambda:
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_cached_assertion (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 83, in test_cached_assertion
self.ldap_conn.sasl_interactive_bind_s("", sasl_creds)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_cached_assertion_unknown_user (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 101, in test_cached_assertion_unknown_user
self.ldap_conn.sasl_interactive_bind_s("", sasl_creds)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

ERROR: test_no_browserid_auth_mech (main.NormalUsageTestCase)

Traceback (most recent call last):
File "/test/functional_test.py", line 177, in test_no_browserid_auth_mech
self.assertRaises(ldap.STRONG_AUTH_NOT_SUPPORTED, lambda:
File "/usr/lib/python2.4/unittest.py", line 320, in failUnlessRaises
callableObj(_args, *_kwargs)
File "/test/functional_test.py", line 177, in
self.assertRaises(ldap.STRONG_AUTH_NOT_SUPPORTED, lambda:
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 196, in sasl_interactive_bind_s
return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls),sasl_flags)
File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 94, in _ldap_call
result = func(args,*kwargs)
AUTH_UNKNOWN: {'info': 'SASL(-4): no mechanism available: No worthy mechs found', 'desc': 'Unknown authentication method'}

Ran 7 tests in 8.397s

FAILED (errors=7)

Inside the chroot, looking at /var/log/messages

Oct 17 23:40:39 dm-hudson01-slave01 python: unable to dlopen /usr/lib/sasl2/libbrowserid.so.0: /usr/lib/sasl2/libbrowserid.so.0: undefined symbol: _plug_get_simple
Oct 17 23:40:39 dm-hudson01-slave01 python: No worthy mechs found
Oct 17 23:40:47 dm-hudson01-slave01 last message repeated 6 times
@ozten
Copy link
Owner Author

ozten commented Oct 25, 2011

bash-3.2# grep _plug_get_simple /usr/lib/*
Binary file /usr/lib/libsasl2.a matches
Binary file /usr/lib/libsasl2.so matches
Binary file /usr/lib/libsasl2.so.2 matches
Binary file /usr/lib/libsasl2.so.2.0.22 matches

Running /sbin/ldconfig doesn't fix the issue.

@ozten
Copy link
Owner Author

ozten commented Oct 25, 2011

Today when trying to repro to document this issue, I see lots of these going to syslog:

Oct 25 16:26:26 dm-hudson01-slave01 audispd: node=dm-hudson01-slave01.mozilla.org type=EOE msg=audit(1319585179.963:3108806):
Oct 25 16:26:26 dm-hudson01-slave01 audispd: queue is full - dropping event
Oct 25 16:26:26 dm-hudson01-slave01 audispd: node=dm-hudson01-slave01.mozilla.org type=EXECVE msg=audit(1319585179.963:3108807): argc=11 a0="ldapmodify" a1="-x" a2="-H" a3="ldap://127.0.0.1:1389/" a4="-D" a5="cn=root,dc=example,dc=com" a6="-w" a7="pass" a8="-a" a9="-f" a10="/test/slapd/users.ldif"
Oct 25 16:26:26 dm-hudson01-slave01 audispd: queue is full - dropping event

I'll file an IT bug, so they know about the issue.

@ozten
Copy link
Owner Author

ozten commented Oct 25, 2011

Filed https://bugzilla.mozilla.org/show_bug.cgi?id=697311 and
https://bugzilla.mozilla.org/show_bug.cgi?id=697313

@fox2mike
Copy link

LDAP servers normally bind to 389 not 1389 (unless you specifically modified it to run at 1389 instead).

@ozten
Copy link
Owner Author

ozten commented Oct 26, 2011

Ya, I did this in-case someone runs a normal LDAP server for another project.

ozten added a commit that referenced this issue Dec 16, 2011
@ozten
Merge pull request #3 from mozilla/remove-syslog
7480f3d 
Removing syslog and using saslplug utils log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ozten @fox2mike