add validation to filters in legal/search endpoint

[pkfec]

Summary (required)

• Resolves Make empty string behavior consistent on the legal search endpoint #6024

• Modify ao_requestor_type filter from int to string

• Add field validation for mur_type filter

• Update tests

• Update docs.py

Required reviewers

2 developers

Impacted areas of the application

• legal/search endpoints

How to test

on local:

• git checkout feature/6024-legal-search-validate-dropdown-filters

• pyenv activate

• run elasticsearch service

• run pytest

• run python cli.py create_index ao_index

• run python cli.py create_index arch_mur_index

• run python cli.py create_index case_index

• run python cli.py load_advisory_opinions 2024-01 (15 AO's loaded to local ES service)

• run python cli.py load_archived_murs

• run python cli.py load_current_murs

• run flask run

• test filters

• http://localhost:5000/v1/legal/search/?type=advisory_opinions (all 15 show)

• http://localhost:5000/v1/legal/search/?type=advisory_opinions&ao_requestor_type=1&ao_requestor_type=6 (8 results show)

• http://localhost:5000/v1/legal/search/?type=advisory_opinions&ao_requestor_type=16&ao_requestor_type=3 (3 results show)

• http://localhost:5000/v1/legal/search/?type=murs&mur_type=current

• http://localhost:5000/v1/legal/search/?type=murs&mur_type=archived

• http://localhost:5000/v1/legal/search/?type="" (should throw validation error)

On Dev: Deploy a feature/test branch and test the filters against Dev api

-test filters:

• https://fec-dev-api.app.cloud.gov/v1/legal/search/?type=advisory_opinions&o_requestor_type=1&ao_requestor_type=6 (238 results)
• https://fec-dev-api.app.cloud.gov/v1/legal/search/?type=advisory_opinions&ao_requestor_type=16&ao_requestor_type=3 (169 results)

[codecov]

Codecov Report

Attention: Patch coverage is 95.65217% with 1 line in your changes missing coverage. Please review.

Project coverage is 88.47%. Comparing base (02b2fbd) to head (97cd455).

Files with missing lines	Patch %	Lines
webservices/filters.py	92.85%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6088      +/-   ##
===========================================
+ Coverage    88.46%   88.47%   +0.01%     
===========================================
  Files           82       82              
  Lines         9091     9110      +19     
===========================================
+ Hits          8042     8060      +18     
- Misses        1049     1050       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cnlucas]

Great job dealing with a tricky issue, and thank you for cleaning up the code!

1. My understanding was FE wants all of the filters to have the same behavior—should we confirm and open a follow-up ticket for the other filters? What about type, ao_is_pending, ao_citation_require_all, case_citation_require_all—can they be updated in this PR since they also have the option in swagger —?
2. If someone includes an empty string and a string it throws a 500—should we update the check_filter_exists function to not throw a 500? IE http://localhost:5000/v1/legal/search/?type=advisory_opinions&ao_requestor_type=1&ao_requestor_type=

[fec-jli]

I agree with @cnlucas.
We should update the check_filter_exists function, something like if len(val) > 0 and '' not in val). Also the check_filter_exists function name is better like filter_validation(kwargs, filter)

[pkfec]

@cnlucas

1. type: existing validation works well and throws 422 error incase an empty string or invalid string type is passed as a parameter which is why i didn't add any additional checks.
   Few examples:

• https://api.open.fec.gov/v1/legal/search/?type=&api_key=DEMO_KEY OR
• https://api.open.fec.gov/v1/legal/search/?type=%22%22&api_key=DEMO_KEY OR
• https://api.open.fec.gov/v1/legal/search/?type=%22asdfasdf%22&api_key=DEMO_KEY

ao_is_pending, ao_citation_require_all, case_citation_require_all: All three are boolean type filters
any value other than True/False, is already giving a 422 error. I don't think additional validation is required on these 3 filters
Few examples:

• https://api.open.fec.gov/v1/legal/search/?type=advisory_opinions&api_key=DEMO_KEY&ao_is_pending
• https://api.open.fec.gov/v1/legal/search/?type=advisory_opinions&api_key=DEMO_KEY&ao_is_pending=test

[pkfec]

@cnlucas

2. ao_requestor_type filter is unique compared to other ones. I have added a custom validation check to return values even an invalid or empty string is passed on to this filter. Thanks for catching 500 error. I fixed the empty string validation error.