Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Charm fails to install from latest/edge, revision 20 #30

Closed
facundofc opened this issue Nov 8, 2023 · 3 comments
Closed

Charm fails to install from latest/edge, revision 20 #30

facundofc opened this issue Nov 8, 2023 · 3 comments

Comments

@facundofc
Copy link

facundofc commented Nov 8, 2023 •
edited
Loading

Bug Description

The charm fails a fresh install from latest/edge channel, revision is 20 at the time of writing.

To Reproduce

juju deploy ubuntu
juju deploy grafana-agent --channel=latest/edge # optionally --revision=20
juju relate ubuntu grafana-agent

charm goes into error state on install hook

Environment

Using juju with an openstack provider with Jammy VMs.

The same exact issue was observed while upgrading from revision 12 to 20 but running on juju with MaaS provider.

Relevant log output

unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install Traceback (most recent call last):
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 41, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from charms.tls_certificates_interface.v2.tls_certificates import (  # type: ignore
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/lib/charms/tls_certificates_interface/v2/tls_certificates.py", line 285, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from cryptography import x509
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/venv/cryptography/x509/__init__.py", line 7, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from cryptography.x509 import certificate_transparency
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/venv/cryptography/x509/certificate_transparency.py", line 11, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from cryptography.hazmat.bindings._rust import x509 as rust_x509
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install ImportError: libssl.so.3: cannot open shared object file: No such file or directory
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install 
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install During handling of the above exception, another exception occurred:
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install 
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install Traceback (most recent call last):
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "./src/charm.py", line 20, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from grafana_agent import METRICS_RULES_SRC_PATH, GrafanaAgentCharm
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/src/grafana_agent.py", line 31, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     from charms.observability_libs.v0.cert_handler import CertHandler
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install   File "/var/lib/juju/agents/unit-grafana-agent-fresh-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 51, in <module>
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install     raise ImportError(
unit-grafana-agent-fresh-0: 14:17:21 WARNING unit.grafana-agent-fresh/0.install ImportError: charms.tls_certificates_interface.v2.tls_certificates is missing; please get it through charmcraft fetch-lib
unit-grafana-agent-fresh-0: 14:17:21 ERROR juju.worker.uniter.operation hook "install" (via hook dispatching script: dispatch) failed: exit status 1
unit-grafana-agent-fresh-0: 14:17:21 DEBUG juju.machinelock created rotating log file "/var/log/juju/machine-lock.log" with max size 10 MB and max backups 5
unit-grafana-agent-fresh-0: 14:17:21 DEBUG juju.machinelock machine lock released for grafana-agent-fresh/0 uniter (run install hook)
unit-grafana-agent-fresh-0: 14:17:21 DEBUG juju.worker.uniter.operation lock released for grafana-agent-fresh/0
unit-grafana-agent-fresh-0: 14:17:21 INFO juju.worker.uniter awaiting error resolution for "install" hook

Additional context

No response
@nishant-dash
Copy link

nishant-dash commented Nov 20, 2023 •
edited
Loading

I tried this on both focal and jammy (principals of ubuntu) and I see the same messages in logs but on the focal version, the install hook fails with that message

ubuntu-focal/0*           active    idle   3        10.5.2.243             
  grafana-agent-fresh/1*  error     idle            10.5.2.243             hook failed: "install"
  
Machine  State    Address     Inst id                               Series  AZ    Message
...
3        started  10.5.2.243  4b643e4a-bfc4-470b-a8c9-92870c883cac  focal   nova  ACTIVE

For jammy, the unit is in a blocked state

ubuntu/1*                 active    idle   2        10.5.0.202             
  grafana-agent/1*        blocked   idle            10.5.0.202             send-remote-write: off, logging-consumer: off, grafana-cloud-config: off

Machine  State    Address     Inst id                               Series  AZ    Message
2        started  10.5.0.202  2af6bb68-a085-4196-a7a7-52dd121c473b  jammy   nova  ACTIVE
...

charm versions I used (ignore the status since ive removed them from my test setup)

App                  Version  Status       Scale  Charm          Channel      Rev  Exposed  Message
grafana-agent                 blocked          1  grafana-agent  latest/edge   20  no       send-remote-write: off, logging-consumer: off, grafana-cloud-config: off
grafana-agent-fresh           error            1  grafana-agent  latest/edge   20  no       hook failed: "install"  
ubuntu               22.04    maintenance      1  ubuntu         stable        24  no       
ubuntu-focal         20.04    maintenance      1  ubuntu         stable        24  no

@sed-i
Copy link
Contributor

sed-i commented Nov 20, 2023

The key line is

ImportError: libssl.so.3: cannot open shared object file: No such file or directory

Similar to #5.

@sed-i
Copy link
Contributor

sed-i commented Nov 22, 2023

Hi @facundofc, @nishant-dash,

  1. The install error is probably because of focal/jammy mismatch between the primary and the subordinate. That's why the libssl.so.3 error.
  2. The send-remote-write: off, logging-consumer: off, grafana-cloud-config: off message is valid and correct: it warns you that key relations are missing.
$ charmcraft status grafana-agent

Track    Base                  Channel    Version    Revision                                                                                                                         
latest   ubuntu 20.04 (amd64)  stable     19         19                                                                                                                               
                               candidate  19         19                                                                                                                               
                               beta       19         19                                                                                                                               
                               edge       21         21                                                                                                                               
         ubuntu 22.04 (amd64)  stable     20         20                                                                                                                               
                               candidate  20         20                                                                                                                               
                               beta       20         20                                                                                                                               
                               edge       20         20

Here's how you can deploy jammy or focal correctly:

juju deploy --channel=edge ubuntu primary-focal --base [email protected]
juju deploy --channel=edge ubuntu primary-jammy --base [email protected]
juju deploy --channel=edge grafana-agent agent-jammy --base [email protected]
juju deploy --channel=edge grafana-agent agent-focal --base [email protected]
juju relate primary-focal agent-focal
juju relate primary-jammy agent-jammy
series: jammy
applications:
  agent-focal:
    charm: grafana-agent
    channel: edge
    revision: 21
    series: focal
  agent-jammy:
    charm: grafana-agent
    channel: edge
    revision: 20
  primary-focal:
    charm: ubuntu
    channel: edge
    revision: 24
    series: focal
    num_units: 1
    to:
    - "0"
    constraints: arch=amd64
    storage:
      block: loop,100M
      files: rootfs,100M
  primary-jammy:
    charm: ubuntu
    channel: edge
    revision: 24
    num_units: 1
    to:
    - "1"
    constraints: arch=amd64
    storage:
      block: loop,100M
      files: rootfs,100M
machines:
  "0":
    constraints: arch=amd64
    series: focal
  "1":
    constraints: arch=amd64
relations:
- - primary-focal:juju-info
  - agent-focal:juju-info
- - primary-jammy:juju-info
  - agent-jammy:juju-info
Model  Controller  Cloud/Region         Version  SLA          Timestamp
ga     lxd         localhost/localhost  3.1.6    unsupported  13:45:14-05:00

App            Version  Status   Scale  Charm          Channel  Rev  Exposed  Message
agent-focal             blocked      1  grafana-agent  edge      21  no       grafana-cloud-config: off, send-remote-write: off, logging-consumer: off
agent-jammy             blocked      1  grafana-agent  edge      20  no       logging-consumer: off, grafana-cloud-config: off, send-remote-write: off
primary-focal  20.04    active       1  ubuntu         edge      24  no    
primary-jammy  22.04    active       1  ubuntu         edge      24  no    

Unit              Workload  Agent  Machine  Public address  Ports  Message
primary-focal/0*  active    idle   0        10.41.168.157           
  agent-focal/0*  blocked   idle            10.41.168.157          grafana-cloud-config: off, send-remote-write: off, logging-consumer: off
primary-jammy/0*  active    idle   1        10.41.168.179           
  agent-jammy/0*  blocked   idle            10.41.168.179          logging-consumer: off, grafana-cloud-config: off, send-remote-write: off

Machine  State    Address        Inst id        Base          AZ  Message
0        started  10.41.168.157  juju-9cc8bd-0  [email protected]      Running
1        started  10.41.168.179  juju-9cc8bd-1  [email protected]      Running

Integration provider     Requirer               Interface              Type         Message
agent-focal:peers        agent-focal:peers      grafana_agent_replica  peer
agent-jammy:peers        agent-jammy:peers      grafana_agent_replica  peer
primary-focal:juju-info  agent-focal:juju-info  juju-info              subordinate
primary-jammy:juju-info  agent-jammy:juju-info  juju-info              subordinate

Potentially related:

Closing for now. Feel free to reopen if the issue is elsewhere.

@sed-i sed-i closed this as completed Nov 22, 2023
@phvalguima phvalguima mentioned this issue Apr 15, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@facundofc @nishant-dash @sed-i