Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable mail MTA/SMTP configuration using MicroProfile Config #9939

Merged
merged 81 commits into from
Mar 26, 2024

Conversation

poikilotherm
Copy link
Contributor

@poikilotherm poikilotherm commented Sep 20, 2023

What this PR does / why we need it:
Instead of setting up the Java Mail manually via asadmin commands, this PR adds an option to configure this the same way we configure other subsystems via MPCONFIG.

Which issue(s) this PR closes:

Resolves container working group milestone C.

Special notes for your reviewer:

  • There are three TODOs where it might make sense to start exposing the support team's name and mail address. Please leave a note how to address these in this PR.

Still missing:

  • Resolve conflicts once OIDC auth: MPCONFIG provisioning, PKCE support and integration tests #9273 is merged
  • Include in test results and coverage once OIDC auth: MPCONFIG provisioning, PKCE support and integration tests #9273 is merged
  • Refactor integration test with nicer way to inject settings from Testcontainers, depending on merge of OIDC auth: MPCONFIG provisioning, PKCE support and integration tests #9273
  • Add startup check once Configurable docroot via MicroProfile Config #9819 is merged (provides the infrastructure)
    • Check that a mail setting has been provided (or warn)
    • Check that for sessions still coming from appserver resources the "from" matches the setting (or warn)
  • Make "system address" configurable via JvmSettings as well, but keep backward compatible with DB setting
  • Add integration test to test sending emails to addresses using UTF-8 characters works
  • Add release note about the backward compatible change where the system address comes from
  • Add second integration test covering user authentication at fake MTA
  • Update documentation how to configure mail
  • Update installer to drop manual setup
  • Update configbaker bootstrap to not set DB setting, add to compose file instead
  • Make new way non-exclusive, try adding backward compatibility for existing setups (less disruptive change)
  • Add release note how to migrate
  • Optional: refresh sessions to pickup live changes
  • Optional: Add unit test to see what happens when creating sessions with invalid data (port, host, from address)

Suggestions on how to test this:
Simply run the tests for the most of it. To see the injection works and the session does what's necessary we could either add an additional API end-to-end test (will require a fake MTA not yet part of API testing!!!) or just take a look in manual testing.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope

Is there a release notes update needed for this change?:
Yes

Additional documentation:
Included

…#7424

We only default to no authentication. We still require people to
configure an SMTP host, only in containers we do default to "smtp"
as a hostname for that (see our compose file).

Username/password cannot have a default and all other special settings
should not be done here. These are highly setup specific.
As a hack to work around testcontainers/testcontainers-java#970,
we add these fake, empty classes. Copied from Spring project.

See also: testcontainers/testcontainers-java#970
…S#7424

Necessary to add some integration testing, verifying sending mails
actually should work.
If not excluded, the very old Javamail 1.4 spec is being used during
local testing, obviously incompatible with Jakarta EE Mail definition.

Exclusion is the only way around this, as we cannot possibly change the
upstream dependencies.
@poikilotherm poikilotherm added Feature: Email Dataset Contact Component: Containers Anything related to cloudy Dataverse, shipped in containers. Size: 3 A percentage of a sprint. 2.1 hours. Feature: Container Guide labels Sep 20, 2023
@coveralls
Copy link

coveralls commented Oct 5, 2023

Coverage Status

coverage: 20.736% (+0.08%) from 20.661%
when pulling af7171e on poikilotherm:7424-maildefinition-ng
into cb3bd0e on IQSS:develop.

@poikilotherm poikilotherm force-pushed the 7424-maildefinition-ng branch from 4111d1d to 2178c83 Compare October 5, 2023 21:13
…7424

Besides adding the JVM option, the logic to receive the setting in MailServiceBean has changed.
The method signature is now returning an optional to enforce the optional nature of the setting.
This replaces the "null" contract from before and requires more changes to code using the lookup.
…dress IQSS#7424

As we changed the lookup function to use Optional<InternetAddress> to enforce the optional
nature of the setting, we now have to change the code using the function.
To ease looking up the (also optional) setting of a support team mail address,
the mail service is extended with another lookup function. This is intended
to replace many manual, error prone lookups, also streamlining the fall-through
behavior when not set, etc.
…#7424

As we now have proper functions to lookup the mail addresses, replace manual lookup and parsing with them.
Document in code how to replace usages, too. (There aren't any,
but in case someone is adding it again in the future, it helps to
have docs)
@pdurbin pdurbin self-assigned this Mar 25, 2024
@pdurbin
Copy link
Member

pdurbin commented Mar 25, 2024

QA notes (I'll update these as I go):

  • This warning is correctly shown in server.log: The :SystemMail DB setting has been deprecated, please reconfigure using JVM option dataverse.mail.system-email|#]

pdurbin and others added 12 commits March 25, 2024 18:00
The setting is already covered by the "host" property string in MailSessionProducer.
As Payara 6.2023.7 still suffers from the MPCONFIG bug where a profiled setting is not easy to override, lets just remove the default for the container profile and make people add it even for containers.
In case people want to debug Jakarta Mail, they activate dataverse.mail.debug. Let's hook into that and add more verbose output from the session producer, too. That way people can make sure everything is set up as they wish.
Also add notes about common ports in use.
@poikilotherm poikilotherm added this to the 6.2 milestone Mar 26, 2024
@pdurbin
Copy link
Member

pdurbin commented Mar 26, 2024

Works great, tests are passing: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-9939/50/testReport/

@poikilotherm and I had to fix a line of code and some docs here and there but we're good now. Merging!

@pdurbin pdurbin merged commit eff5188 into IQSS:develop Mar 26, 2024
18 of 19 checks passed
@pdurbin pdurbin removed their assignment Mar 26, 2024
@poikilotherm poikilotherm deleted the 7424-maildefinition-ng branch March 27, 2024 07:01
@pdurbin pdurbin changed the title Enable mail MTA configuration using MicroProfile Config Enable mail MTA/SMTP configuration using MicroProfile Config Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Containers Anything related to cloudy Dataverse, shipped in containers. Feature: Container Guide Feature: Email Dataset Contact Size: 10 A percentage of a sprint. 7 hours.
Projects
None yet
4 participants