Skip to content

Commit

Permalink
Filter out some infrastructure details from Error API Responses (#1412)
Browse files Browse the repository at this point in the history
<!-- Specify linked issues and REMOVE THE UNUSED LINES -->

 Close advthreat/iroh#8471

Clear some infrastructure details from Bundle Import API response errors.
Aditionally, attempt to reduce some verbosity in the logs.

<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="iroh-services-clients">[§](#iroh-services-clients)</a> IROH Services Clients
=====================================================================================

Put all informations that need to be communicated to IROH Services Clients.
Typically IROH-UI, ATS Integration, Orbital, etc...
 -->

<a name="qa">[§](#qa)</a> QA
============================

Describe the steps to test your PR.

Issue some regression tests on the Bundle Import API.

<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="ops">[§](#ops)</a> Ops
===============================

  Specify Ops related issues and documentation
- Config change needed: threatgrid/tenzin#
- Migration needed: threatgrid/tenzin#
- How to enable/disable that feature: (ex remove service from `bootstrap.cfg`, add scope to org)
-->

<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="documentation">[§](#documentation)</a> Documentation
=============================================================

  Public Facing documentation section;
- Public documentation updated needed: threatgrid/iroh-ui#
  See internal [doc file](./services/iroh-auth/doc/public-doc.org)
 -->

<a name="release-notes">[§](#release-notes)</a> Release Notes
=============================================================

<!-- REMOVE UNUSED LINES -->

```
intern: filter out some infrastructure details fron api error responses
```

<a name="squashed-commits">[§](#squashed-commits)</a> Squashed Commits
======================================================================
  • Loading branch information
gbuisson authored Feb 27, 2024
1 parent 08be33c commit a584b6d
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 22 deletions.
32 changes: 20 additions & 12 deletions src/ctia/bundle/core.clj
Original file line number Diff line number Diff line change
Expand Up @@ -274,12 +274,22 @@
not-submitted)))
bundle-import-data))

(defn obfuscate-errors
"Given a Bundle import intermediate response,
attempt to obfuscate any leaky error"
[{:keys [error] :as res}]
(cond-> res
(map? error) (assoc :error (select-keys error [:type]))))

(s/defn build-response :- BundleImportResult
"Build bundle import response"
[bundle-import-data :- BundleImportData]
{:results (map
#(dissoc % :new-entity :old-entity)
(apply concat (vals bundle-import-data)))})
[bundle-import-data :- BundleImportData
obfuscate-errors? :- s/Bool]
{:results
(->> (vals bundle-import-data)
(apply concat)
(map #(cond-> (dissoc % :new-entity :old-entity)
obfuscate-errors? obfuscate-errors)))})

(defn bulk-params [get-in-config]
{:refresh
Expand All @@ -290,8 +300,7 @@
(let [errors (->> response
:results
(filter :error))]
(doseq [error errors]
(log/warn error)))
(log/warn (take 10 errors)))
response)

(defn entity->bundle-keys
Expand Down Expand Up @@ -333,12 +342,11 @@
tempids (->> bundle-import-data
(map (fn [[_ entities-import-data]]
(entities-import-data->tempids entities-import-data)))
(apply merge {}))]
(debug "Import bundle response"
(->> (bulk/create-bulk bulk tempids auth-identity (bulk-params get-in-config) services)
(with-bulk-result bundle-import-data)
build-response
log-errors))))
(apply merge {}))
bulk-res (->> (bulk/create-bulk bulk tempids auth-identity (bulk-params get-in-config) services)
(with-bulk-result bundle-import-data))]
(log-errors (build-response bulk-res false))
(build-response bulk-res true)))

(defn bundle-max-size [get-in-config]
(bulk/get-bulk-max-size get-in-config))
Expand Down
20 changes: 10 additions & 10 deletions test/ctia/bundle/routes_test.clj
Original file line number Diff line number Diff line change
Expand Up @@ -413,20 +413,20 @@
:body bundle
:headers {"Authorization" "45c1f5e3f05d0"})
bundle-result-create (:parsed-body response-create)]
(clojure.pprint/pprint bundle-result-create)
(when (is (= 200 (:status response-create)))
(is (every? #(= "created" %)
(->> (:results bundle-result-create)
(filter #(= "sighting" %))
(map :result)))
"All valid entities are created")
(is (every? #(nil? (:index %))
(map :error
(:results bundle-result-create)))
"No ES infrastructure details are exposed in the errors")
(doseq [entity (:sightings bundle)]
(validate-entity-record
app
(find-result-by-original-id bundle-result-create (:id entity))
entity))
app
(find-result-by-original-id bundle-result-create (:id entity))
entity))
(let [indicators (filter
#(= :indicator (:type %))
(:results bundle-result-create))]
#(= :indicator (:type %))
(:results bundle-result-create))]
(is (seq indicators)
"The result collection for indicators is not empty")
(is (every? #(contains? % :error) indicators)))))
Expand Down

0 comments on commit a584b6d

Please sign in to comment.