Abschluss Arbeitspaket 2 IWM

README.md

@@ -1,104 +1,34 @@
 # ontodocker
 
-Prerequisites: Docker and docker-compose 
+Prerequisites: Docker and docker compose 
 
-### Keycloak installation and configurations
-
- **You must configure Keycloak first!**
-
-Go to the keycloak directory, edit password in `Dockerfile` and `docker-compose.yml`.
-
-Start Keycloak container with
-```bash
-docker-compose up -d --build
-```
-in the terminal (cmd)
-
-check the status
-```bash
-docker-compose logs keycloak
-```
-
-After the start, (You have to wait until Keycloak is completely ready and you'll see the line `Running the server in development mode. DO NOT use this configuration in production.` in terminal.)
-
-go to http://localhost:8080, enter the admin password set in the `Dockerfile`, then you'll see the homepage of Keycloak
-
-![Keycloak Homepage](source/images/keycloak_homepage.PNG "Keycloak Homepage")
-
-Now we need to create a client for our application.
-
-1. Fill in the basic information, e.g. email, name (we need this). And then Save
-![Keycloak Admin User](source/images/keycloak_admin_user.PNG "Keycloak Admin User Edit")
-
-2. Create client for our application
-![Keycloak Create Client](source/images/keycloak_client.PNG "Keycloak Create Client")
-![Keycloak Create Client2](source/images/keycloak_client2.PNG "Keycloak Create Client2")
-![Keycloak Create Client3](source/images/keycloak_client3.PNG "Keycloak Create Client3")
-3. Get client secret (client secret is required .env)
-![Keycloak Client Secret](source/images/keycloak_client_secret.PNG "Keycloak Client Secret")
-4. Use roles to control access to the application
-
-    create Realm roles: `App-admin`, `App-insider`, `App-guest`
-
-    (If you don't know what roles mean in Keycloak, see https://stackoverflow.com/questions/47837613/how-are-keycloak-roles-managed)
-
-    ![Keycloak Create Role](source/images/keycloak_role1.PNG "Keycloak Create Role")
-    ![Keycloak Create Role](source/images/keycloak_role2.PNG "Keycloak Create Role")
-    ![Keycloak Create Role](source/images/keycloak_role3.PNG "Keycloak Create Role")
-
-    Now we're going to assign roles to users, e.g., assign `App-insider` role to user `test_insider`
-    ![Keycloak Users](source/images/keycloak_role4.PNG "Keycloak Users")
-    ![Keycloak Assign Role](source/images/keycloak_role5.PNG "Keycloak Assign Role")
-    As you can see, the "admin" role is already created by keycloak by default. The `App-admin` role was created for demo purposes and has the same effect as the `admin` role.
-
-    Make sure that client scope `roles` is assigned to the client `glass`, if not, add the client scope `roles` to `glass`
-    ![Keycloak Client Scopes](source/images/keycloak_role6.PNG "Keycloak Client Scopes")
-    Now go to Client scopes (left panel), enter roles page:
-5.
-    ![Keycloak Client Scopes](source/images/keycloak_role7.PNG "Keycloak Client Scopes")
-    ![Keycloak Client Scopes](source/images/keycloak_role8.PNG "Keycloak Client Scopes")
-    modify `realm roles` & `client roles` as below:
-    ![Keycloak Client scope details](source/images/keycloak_role9.PNG "Keycloak Client scope details")
-    ![Keycloak realm roles](source/images/keycloak_role10.PNG "Keycloak realm roles")
-    ![Keycloak client roles](source/images/keycloak_role11.PNG "Keycloak client roles")
-    Apparently we only use the realm roles, but it can't hurt to set client roles together
-
-
-**Now you have done the Keycloak configuration part!**
-
-### ontodocker installation
+### Ontodocker installation
 
 Once you have done the keycloak configuration part. Go back to the parent directory (`cd ..`)
 
 Create a `.env` file with the following contents and fill or change them accordingly.
 ```
 ONTODOCKER_RUN_PORT=8000
-ADMIN_EMAIL=
 JWT_SECRET_KEY=
 JWT_DEFAULT_DAYS_VALID=90
 JWT_MIN_DAYS_VALID=1
 JWT_MAX_DAYS_VALID=90
 MAX_SESSION_TIME_IN_DAYS=14
 FUSEKI_ADMIN_USER=admin
 FUSEKI_ADMIN_PW=changeme
-APP_URI=http://fastapi:8000
-KEYCLOAK_HOST=http://keycloak:8080
-KEYCLOAK_REALM=master
-KEYCLOAK_CLIENT_ID=ontodocker
-KEYCLOAK_CLIENT_SECRET=
-KEYCLOAK_ROLES_ADMIN=App-admin,admin
-KEYCLOAK_ROLES_RW=App-insider
-KEYCLOAK_ROLES_RO=App-guest
 ALLOW_UNAUTHORIZED_READONLY_API_ACCESS=false
 ALLOW_UNAUTHORIZED_READONLY_UI_ACCESS=true
+ANONYMOUS_IS_ADMIN=false
 ```
 
 Create a random key for `JWT_SECRET_KEY`   by excecuting
 ```
 openssl rand -hex 36
 ```
-in a command line and fill the line in the `.env` file.
+in a command line and fill the line in the `.env` file or use another randomized string.
 
+Important: If you start the application for the first time set `ANONYMOUS_IS_ADMIN` to `true` and `ALLOW_UNAUTHORIZED_READONLY_UI_ACCESS` to `true`. That will give you administration access to configure local users or SSO Providers (like Keycloak (description below)) in the Administration interface.
+If you only need a local developing instance you can also fully use the anonymous user except for saving SPARQL queries.
 
 Edit the Fuseki admin password (`ADMIN_PASSWORD`) as well and `JAVA_OPTIONS` for Java Virtual Machine (JVM) memory settings in `docker-compose-dev.yml`.
 
@@ -109,24 +39,54 @@ ln -s docker-compose-dev.yml docker-compose.yml
 
 Build the docker container with
 ```bash
-docker-compose build
+docker compose build
 ```
 
 Start the docker container with
 ```bash
-docker-compose up -d 
+docker compose up -d 
 ```
 
 Watch the logs with
 ```bash
-docker-compose logs -f
+docker compose logs -f
 ```
 Now you may go to http://localhost:8000
 
+### Keycloak installation and configurations (optional)
+
+Go to the keycloak directory, edit password in `Dockerfile` and `docker-compose.yml`. And watch for the KC_HOSTNAME_URL if you plan to access keycloaks admin interface from a different IP.
+
+Start Keycloak container with
+```bash
+docker compose up -d --build
+```
+in the terminal (cmd)
+
+check the status
+```bash
+docker compose logs -f keycloak
+```
+
+After the start, (You have to wait until Keycloak is completely ready and you'll see the line `Running the server in development mode. DO NOT use this configuration in production.` in terminal.)
+
+go to http://localhost:8080 (or your IP where you installed the keycloak), enter the admin password set in the `Dockerfile`, then you'll see the homepage of Keycloak
+
+![Keycloak Homepage](source/images/keycloak_homepage.PNG "Keycloak Homepage")
+
+Now we need to create a client for our application.
+
+
+1. Create client for your application (in screenshot the client id is `glass` but you can name it whatever you want).
+Also be sure that the valid redirect URL contains the port (default e.g. 8000, like `http://{yourip}:8000/*`)
+![Keycloak Create Client](source/images/keycloak_client.PNG "Keycloak Create Client")
+![Keycloak Create Client2](source/images/keycloak_client2.PNG "Keycloak Create Client2")
+![Keycloak Create Client3](source/images/keycloak_client3.PNG "Keycloak Create Client3")
+3. Get client secret
+![Keycloak Client Secret](source/images/keycloak_client_secret.PNG "Keycloak Client Secret")
 
+The .well-known URL is normally `http://{your_keycloak_ip}:8080/realms/master/.well-known/openid-configuration`
+Apparently we only use the realm roles, but it can't hurt to set client roles together
 
-## Authors & Acknowledgment
-Robert Heimsoth (DECOIT GmbH & Co. KG)
-Jannis Grundmann (Leibniz-Institut für Werkstofforientierte Technologien - IWT) 
 
-Based on FastOntodocker by Ya-Fan Chen ([email protected])
+**Now you have done the Keycloak configuration!**

data/fuseki/configuration/assembler.ttl

@@ -18,7 +18,7 @@
     fuseki:serviceUpdate     "update" ;    # SPARQL update service
     fuseki:serviceUpload     "upload" ;    # Non-SPARQL upload service
     fuseki:serviceReadWriteGraphStore "data" ;     # SPARQL Graph store protocol
-    # ###################################################################################################################fuseki:serviceReadGraphStore    "data" ;     # SPARQL Graph store protocol (read only)
+    # #####################################################################################################################################################fuseki:serviceReadGraphStore    "data" ;     # SPARQL Graph store protocol (read only)
     fuseki:dataset           <#text> ;
 #    fuseki:dataset           <#tdb> ;
     .

docker-compose-dev.yml

@@ -17,7 +17,7 @@ services:
       - testo
       #- default
     ports:
-      - "8000:${ONTODOCKER_RUN_PORT}"
+      - "${ONTODOCKER_RUN_PORT}:8000"
       - 5678:5678
 #    restart: on-failure
 

example/example.ipynb

@@ -54,7 +54,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -81,7 +81,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "metadata": {
     "collapsed": false,
     "pycharm": {
@@ -92,11 +92,8 @@
    "source": [
     "ontodocker_url = \"http://localhost:8000\"\n",
     "\n",
-    "# Admin Token\n",
-    "ontodocker_jwt = \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJNYXRlcmlhbERpZ2l0YWwiLCJpYXQiOjE3MTc0MDMwNjcuODY0NzA0LCJleHAiOjE3MjUxNzkwNjcuODY0NzA0LCJhdWQiOiJvbnRvZG9ja2VyIiwibmFtZSI6IlJvYmVydCBIZWltc290aCIsImVtYWlsIjoiaGVpbXNvdGhAZGVjb2l0LmRlIiwicm9sZSI6WyJkZWZhdWx0LXJvbGVzLW1hc3RlciIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iLCJBcHAtYWRtaW4iXX0.MNdTCPD1HxWlF1yBskryrXyEhfmV5QyTbKsc99znP48\"\n",
-    "\n",
-    "# Guest Token\n",
-    "# ontodocker_jwt = \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJNYXRlcmlhbERpZ2l0YWwiLCJpYXQiOjE3MTczOTYzNzguMjMyNjY1LCJleHAiOjE3MjUxNzIzNzguMjMyNjY1LCJhdWQiOiJvbnRvZG9ja2VyIiwibmFtZSI6Ik9udG8gR3Vlc3QiLCJlbWFpbCI6Im9udG9ndWVzdEBvbnRvZ3Vlc3QuZGUiLCJyb2xlIjpbImRlZmF1bHQtcm9sZXMtbWFzdGVyIiwiQXBwLWd1ZXN0Iiwib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiJdfQ.1B0UyDpb5dE01fmmnTe6iFo1yBzsAXfCP5dYG4do3xk\"\n",
+    "# Token\n",
+    "ontodocker_jwt = \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJNYXRlcmlhbERpZ2l0YWwiLCJpYXQiOjE3MjM0NjYwMzMuNjQ5ODg1LCJleHAiOjE3MzEyNDIwMzMuNjQ5ODg1LCJhdWQiOiJvbnRvZG9ja2VyIiwidXNlcmlkIjoxfQ.Ji7whjr7wTBRVhbi-X4neFvvOtyRhZeDXg57M7KPgzs\"\n",
     "\n",
     "headers = {\"Authorization\": f\"Bearer {ontodocker_jwt}\"}\n",
     "\n",
@@ -114,14 +111,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[\"http://localhost:8000/api/jena/test_dataset/sparql\",\"http://localhost:8000/api/jena/test/sparql\"]\n"
+      "[\"http://localhost:8000/api/jena/test_dataset/sparql\",\"http://localhost:8000/api/jena/leeres_dataset/sparql\",\"http://localhost:8000/api/jena/test/sparql\"]\n"
      ]
     }
    ],
@@ -138,14 +135,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\"Name already registered /test_dataset\\n\"\n"
+      "\"401 - Unauthorized\"\n"
      ]
     }
    ],

fastapi_app/app/config.py

@@ -14,34 +14,21 @@
 os.makedirs(web_dir, exist_ok=True)
 
 class Settings(BaseSettings):
-    SQLALCHEMY_DATABASE_URI: str = f"sqlite:///{os.path.join(user_dir, 'user.db')}"
+    SQLALCHEMY_DATABASE_URI: str = f"sqlite:///{os.path.join(user_dir, 'ontodocker.db.sqlite')}"
 
     JWT_SECRET_KEY: str = os.environ.get("JWT_SECRET_KEY", str(secrets.token_urlsafe(16)))#  # use openssl rand -hex 48
     JWT_DEFAULT_DAYS_VALID: int = int(os.environ.get("JWT_DEFAULT_DAYS_VALID", "1"))
     #authjwt_secret_key = JWT_SECRET_KEY
-
-    # roles that are allowed to access, go to Keycloak console to configure if needed
-    KEYCLOAK_ADMIN_ROLES: list = [x.strip() for x in os.environ.get("KEYCLOAK_ROLES_ADMIN", "").split(",")]
-    KEYCLOAK_READWRITE_ROLES: list = [x.strip() for x in os.environ.get("KEYCLOAK_ROLES_RW", "").split(",")]
-    KEYCLOAK_READONLY_ROLES: list = [x.strip() for x in os.environ.get("KEYCLOAK_ROLES_RO", "").split(",")]
-    KEYCLOAK_REQUIRED_ROLES: list = KEYCLOAK_ADMIN_ROLES + KEYCLOAK_READWRITE_ROLES + KEYCLOAK_READONLY_ROLES
-    ADMIN_EMAIL: str = os.environ.get("ADMIN_EMAIL", "")  # admin email, used for API Key authenticate
+
+    OIDC_ADMIN_ROLE: str = "admin"
+    OIDC_READWRITE_ROLE: str = "rw"
+    OIDC_READONLY_ROLE: str = "ro"
+    OIDC_REQUIRED_ROLES: list = [OIDC_ADMIN_ROLE, OIDC_READWRITE_ROLE, OIDC_READONLY_ROLE]
 
 @AuthJWT.load_env
 def get_config():
     return Settings()
 
-class KeycloakSettings(BaseSettings):
-    host: str = os.environ.get("KEYCLOAK_HOST", "")  # This is your Keycloak URL (use http://<IPv4 address>:8080) (Don't use VPN)
-    realm: str = os.environ.get("KEYCLOAK_REALM", "") # This is your Keycloak Realm name, default is "master"
-    client_id: str = os.environ.get("KEYCLOAK_CLIENT_ID", "") # This is your Keycloak client id that you have previously created
-    client_secret: str = os.environ.get("KEYCLOAK_CLIENT_SECRET", "") # This is your Keycloak client secret
-    app_uri: str = os.environ.get("APP_URI", "") # This is your application URL
-    token_uri: str = f"{host}/realms/{realm}/protocol/openid-connect/token"  # http://<IPv4 address>:8080/realms/master/protocol/openid-connect/token
-    scope: str = "openid email profile"
-    verify: bool = True
-
-
 class FusekiSettings(BaseSettings):
     FUSEKI_ADMIN_USER = os.environ.get("FUSEKI_ADMIN_USER", "")
     FUSEKI_ADMIN_PW = os.environ.get("FUSEKI_ADMIN_PW", "") 
@@ -53,11 +40,6 @@ def get_settings():
     return Settings()
 
 
-@lru_cache()
-def get_keycloak_settings():
-    return KeycloakSettings()
-
-
 @lru_cache()
 def get_fuseki_settings():
     return FusekiSettings()