Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ED-2793] Fix installation issues #3903

Closed
wants to merge 75 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
634bb0a
Devops changes for Program join stories (ED-599) && Block Level Filte…
VISHNUDAS-tunerlabs Apr 27, 2023
b626c00
Fix syntax for API getSolutionReportInformation
aks30 May 3, 2023
1406fd0
Merge pull request #3799 from aks30/patch-5
santhosh-tg May 3, 2023
e049397
ml-reports env syntax error fix added
VISHNUDAS-tunerlabs May 11, 2023
5b13f74
Merge pull request #3815 from VISHNUDAS-tunerlabs/env_syntax_fix_ml_r…
santhosh-tg May 11, 2023
2f67f7a
ED-1945 feat:onboarded Question-set,question copy api
princegupta1131 May 15, 2023
9cdd587
programuser.info Kafka topic added to script
VISHNUDAS-tunerlabs May 15, 2023
595adac
Merge pull request #3821 from VISHNUDAS-tunerlabs/add-Kafka-topic-Scr…
tsprasath May 15, 2023
3e65782
ED-1945 feat:onboarded Question-set,question copy api (#3820)
princegupta1131 May 18, 2023
2998eec
Update main.yml
spsac08 May 18, 2023
097bb88
Merge pull request #3823 from spsac08/release-6.0.0
santhosh-tg May 19, 2023
c5687dc
#SB-0000 fix: updating the pdataid and version
princegupta1131 Jun 6, 2023
e3811c2
Merge branch 'release-6.0.0' of github.com:project-sunbird/sunbird-de…
princegupta1131 Jun 6, 2023
17fa7c1
Merge pull request #3836 from princegupta1131/release-6.0.0
santhosh-tg Jun 6, 2023
5d52491
Added ML PII Data Product job to submit APO
Shakthieshwari Jun 6, 2023
3bc2451
Merge pull request #3837 from shikshalokam/release-6.0.0
santhosh-tg Jun 8, 2023
217e1d2
Add variable forreport automation
Ashwiniev95 Jun 19, 2023
f5c211d
Issue #ED-0000 fix:added env variable to store default board value
abhishekpnt Jun 20, 2023
5266249
Issue #ED-0000 fix:added fallback for default board
abhishekpnt Jun 20, 2023
ff5b2d3
Added env key for timezone Difference
Jun 20, 2023
aba7e5f
Added env key for timezone Difference
Jun 20, 2023
cfe06e6
Added env key for timezone Difference
Jun 20, 2023
a8a9d76
Merge pull request #3848 from ankitpws/addedTimeZoneOffsetEnvKey
santhosh-tg Jun 20, 2023
ddc31dc
Merge branch 'release-6.0.0' of https://github.com/project-sunbird/su…
Ashwiniev95 Jun 20, 2023
696ab85
update variables
Ashwiniev95 Jun 20, 2023
c3e0f45
remove a variable
Ashwiniev95 Jun 20, 2023
5f3a35f
remove the variable
Ashwiniev95 Jun 20, 2023
2b913e0
Remove a variable
Ashwiniev95 Jun 21, 2023
d716a07
Merge pull request #3847 from abhishekpnt/release-6.0.0
santhosh-tg Jun 22, 2023
711c106
Merge branch 'release-6.0.0' of https://github.com/project-sunbird/su…
Ashwiniev95 Jun 22, 2023
42b2bd7
Remove a variable
Ashwiniev95 Jun 22, 2023
f99495c
Added base url to config
adithyadinesh0412 Jun 22, 2023
b6efe59
Merge pull request #4 from adithyadinesh0412/release-6.0.0
aks30 Jun 22, 2023
d9ffacc
Update main.yml
aks30 Jun 22, 2023
1d7ae76
Update main.yml
aks30 Jun 26, 2023
b04cc05
Add reference for pvt repo values
aks30 Jun 26, 2023
94de8aa
Add reference for pvt repo secret values
aks30 Jun 26, 2023
2224063
Merge pull request #3849 from shikshalokam/release-6.0.0
santhosh-tg Jun 26, 2023
4e59b60
Update common.yml
aks30 Jun 26, 2023
140485a
Update secrets.yml
aks30 Jun 26, 2023
6e3c703
Merge pull request #3854 from shikshalokam/release-6.0.0
santhosh-tg Jun 26, 2023
a0a630f
Remove space for 6.0 added variables
aks30 Jun 30, 2023
c05302f
Remove extra spaces
aks30 Jun 30, 2023
6dfd079
Update main.yml
aks30 Jun 30, 2023
76ccefb
Merge pull request #3861 from shikshalokam/release-6.0.0
santhosh-tg Jun 30, 2023
f2f6a80
Update main.yml
aks30 Jun 30, 2023
15182e1
Merge pull request #3862 from shikshalokam/release-6.0.0
santhosh-tg Jun 30, 2023
ac256ea
Update config.j2
spsac08 Jul 3, 2023
11a9abc
Merge pull request #6 from spsac08/patch-1
aks30 Jul 3, 2023
4eef8d0
Merge pull request #3864 from shikshalokam/release-6.0.0
santhosh-tg Jul 3, 2023
d116ed4
Issue #ED-2580 fix: Update default tenant
rajeshkumaravel Jul 10, 2023
d83a348
Issue #ED-2580 fix: Update default tenant
rajeshkumaravel Jul 10, 2023
5c1c1a4
Merge pull request #3867 from rajeshkumaravel/release-6.0.0
santhosh-tg Jul 10, 2023
8494c44
Issue fix ED-2586 'Diksha' text is appearing in help page and in SSO …
Jul 19, 2023
7cc1432
Merge pull request #3869 from rajeevsathish/r6.0.0
santhosh-tg Jul 19, 2023
d165f00
[ED-2164] Release 6.0.0 changes (#3840)
santhosh-tg Jul 26, 2023
4df16ff
Issue #ED-2632 fix: removed unused env variables and added OCI endpoi…
abhishekpnt Aug 9, 2023
da25f00
ENV changes related to CSP (#3878)
VISHNUDAS-tunerlabs Aug 14, 2023
100ef92
provider variable name change (#3880)
VISHNUDAS-tunerlabs Aug 16, 2023
f484132
#ED-1383 Release 6.0.0 (#3875)
spsac08 Aug 16, 2023
7220872
changed to public variables for cloud config
VISHNUDAS-tunerlabs Aug 16, 2023
67f640c
Merge pull request #3882 from VISHNUDAS-tunerlabs/CSP-Public-Bucket
santhosh-tg Aug 16, 2023
6d6861e
CSP ML Analytics Changes
adithyadinesh0412 Aug 21, 2023
2859816
Merge pull request #3884 from adithyadinesh0412/csp_ml_analytics_changes
santhosh-tg Aug 22, 2023
af22c69
ED-1251 - Add OCI specific devops changes - superseeds PR 3817 , 3735…
santhosh-tg Aug 22, 2023
d2b113d
Issue #ED-2632 fix: removed duplicate variables (#3887)
abhishekpnt Aug 22, 2023
9221672
Issue #LR-539 feat: configuration reconcile (#3888)
Jayaprakash8887 Aug 23, 2023
408c8c4
Fix syntax (#3894)
santhosh-tg Sep 14, 2023
d6c6875
Updated vars for cron schedule (#3895)
raghupathiguduri Sep 19, 2023
8dc81f6
[ED-2793] Fix: release-6.0.0 installation issues (#3900)
santhosh-tg Sep 25, 2023
6320664
Update main.yml with cassandra (#3845) (#3901)
santhosh-tg Sep 25, 2023
6ce00cf
ED-2838: ##Release-6.0.0 new variables (#3899)
tsprasath Sep 26, 2023
c1b8c59
Update maven repo url
santhosh-tg Sep 28, 2023
7ae3271
Add missing jobs
santhosh-tg Sep 28, 2023
22f4afa
Fix docker creds secrets issue when having specialcharacters
santhosh-tg Sep 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions ansible/artifacts-download.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,16 @@
aws_access_key_id: "{{ cloud_artifact_storage_accountname }}"
aws_secret_access_key: "{{ cloud_artifact_storage_secret }}"
when: cloud_service_provider == "aws"

- name: download artifact from oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: download.yml
vars:
local_file_or_folder_path: "{{ artifact_path }}"
oss_bucket_name: "{{ cloud_storage_artifacts_bucketname }}"
oss_path: "{{ artifact }}"
when: cloud_service_provider == "oci"
13 changes: 13 additions & 0 deletions ansible/artifacts-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,16 @@
aws_access_key_id: "{{ cloud_artifact_storage_accountname }}"
aws_secret_access_key: "{{ cloud_artifact_storage_secret }}"
when: cloud_service_provider == "aws"

- name: upload artifact to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload.yml
vars:
local_file_or_folder_path: "{{ artifact_path }}"
oss_bucket_name: "{{ cloud_storage_artifacts_bucketname }}"
oss_path: "{{ artifact }}"
when: cloud_service_provider == "oci"
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ noAccount=Don''t have an account?
username=Username or Mobile number
goBack=<<Go Back
mergeAccountMessage=Enter Mobile number / Email Address OR use Login with Google to identify the account from which you want to merge usage details
migrateAccountMessage=Confirm the password for the DIKSHA account you want to merge of click Login with Google to sign in using your Gmail account
migrateAccountMessage=Confirm the password for the SUNBIRD account you want to merge of click Login with Google to sign in using your Gmail account
inCorrectPasswordError=The password entered is incorrect. Enter the password again.
emailOrPhone=Email Address / Mobile Number
placeholderForEmailOrPhone=Enter Email Address / Mobile Number
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2477,17 +2477,14 @@ if (typeof module != 'undefined') {

let pdataId = "";
let tenantSlug;
if (window.location.origin.indexOf("diksha.gov.in") >= 0) {
pdataId = "prod.diksha.portal";
} else if (window.location.origin.indexOf("staging.ntp.net.in") >= 0) {
pdataId = "staging.diksha.portal";
} else if (window.location.origin.indexOf("staging.sunbirded.org") >= 0) {
if (window.location.origin.indexOf("staging.sunbirded.org") >= 0) {
pdataId = "staging.diksha.portal";
} else if (window.location.origin.indexOf("dev.sunbirded.org") >= 0) {
pdataId = "dev.sunbird.portal";
tenantSlug = "sunbird";
} else {
pdataId = "preprod.diksha.portal";
tenantSlug = "sunbird";
pdataId = "dev.sunbird.portal";
}

let curUrlObj = window.location;
Expand All @@ -2506,7 +2503,7 @@ if(client_id.toLowerCase() === 'android'){
"telemetry": {
"pdata": {
"id": pdataId,
"ver": "5.2.0",
"ver": "6.0.0",
"pid": "sunbird-portal"
}
}
Expand All @@ -2533,7 +2530,7 @@ if(client_id.toLowerCase() === 'android'){
request: {
filters: {
isTenant: true,
slug: id || 'ntp'
slug: id || 'sunbird'
}
}
}),
Expand Down
28 changes: 28 additions & 0 deletions ansible/assets-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,31 @@
vars:
local_file_or_folder_path: "{{ assets }}/*"
when: cloud_service_provider == "gcloud"

##### OCI
- name: this block consists of tasks related to oci oss
block:
- name: set common oci variables
set_fact:
oss_bucket_name: "{{ cloud_storage_playercdn_bucketname }}"
oss_path: ""
oss_namespace: "{{oci_namespace}}"

- name: delete files and folders from oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: delete-folder.yml

- name: upload batch of files to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
local_file_or_folder_path: "{{ assets }}"
when: cloud_service_provider == "oci"
79 changes: 78 additions & 1 deletion ansible/deploy-plugins.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@
local_file_or_folder_path: "{{ source_file_name }}"
tags:
- preview

- block:
- name: delete and re-upload plugins
include_role:
Expand Down Expand Up @@ -222,3 +221,81 @@
tags:
- plugins
when: cloud_service_provider == "aws"


################################### OCI tasks #########################
- name: this block consists of tasks related to oci oss
block:
- name: set common oci variables
set_fact:
oss_bucket_name: "{{ cloud_storage_content_bucketname }}"
oss_namespace: "{{ oci_namespace }}"
tags:
- always

- block:
- name: delete files and folders from oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: delete-folder.yml
vars:
oss_path: "{{ folder_name }}/"
tags:
- content-editor
- collection-editor
- generic-editor
- preview

- block:
- name: upload folder to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_path: "{{ folder_name }}/"
local_file_or_folder_path: "{{ source_name }}"
tags:
- content-editor
- collection-editor
- generic-editor
- preview
- editor
- core-plugins

- block:
- name: upload file to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload.yml
vars:
oss_path: "artefacts/content-player/content-player-{{ player_version_number }}.zip"
local_file_or_folder_path: "{{ source_file_name }}"
tags:
- preview

- block:
- name: delete and re-upload plugins for oci
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: "{{ item[0] }}"
vars:
object_prefix: "content-plugins/{{ item[1] }}/"
local_file_or_folder_path: "{{ source_folder }}/{{ item[1] }}"
with_nested:
- ['oss-delete-batch-no-poll.yml', 'oss-upload-batch-no-poll.yml']
- "{{ lookup('file', plugins_to_delete_and_upload).split('\n') }}"
tags:
- plugins
when: cloud_service_provider == "oci"
82 changes: 81 additions & 1 deletion ansible/desktop-faq-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@
- upload-csv-template

- block:
- name: upload folder to aws s3 - discussion-ui
- name: upload folder to aws s3 - discusion-ui
include_role:
name: aws-cloud-storage
tasks_from: upload-folder.yml
Expand All @@ -222,3 +222,83 @@
tags:
- upload-discussion-ui
when: cloud_service_provider == "aws"


##### OCI Tasks ####

- name: this block consists of tasks related to oci oss
block:
- name: set common oci variables
set_fact:
local_file_or_folder_path: "{{ playbook_dir }}/../{{ src_file_path }}"
tags:
- always

- block:
- name: upload file to oci oss public bucket
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload.yml
vars:
oss_bucket_name: "{{ cloud_storage_public_bucketname }}"
tags:
- upload-desktop-faq

- block:
- name: upload file to oci oss private bucket
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload.yml
vars:
oss_bucket_name: "{{ cloud_storage_label_bucketname }}"
tags:
- upload-label

- block:
- name: upload folder to oci oss - chatbot
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_path: "{{ destination_path }}/"
oss_bucket_name: "{{ cloud_storage_chatbot_bucketname }}"
tags:
- upload-chatbot-config

- block:
- name: upload folder to oci oss - csv-template
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_path: "{{ destination_path }}/"
oss_bucket_name: "{{ cloud_storage_sourcing_bucketname }}"
tags:
- upload-csv-template

- block:
- name: upload folder to oci oss - discussion-ui
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_path: "{{ destination_path }}/"
oss_bucket_name: "{{ cloud_storage_discussionui_bucketname }}"
tags:
- upload-discussion-ui
when: cloud_service_provider == "oci"
4 changes: 4 additions & 0 deletions ansible/es.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
when: cloud_service_provider == "aws"
- role: es-gcs-snapshot
when: cloud_service_provider == "gcloud"
- role: es-oci-snapshot
when: cloud_service_provider == "oci"
- role: es5-snapshot-purge
tags:
- es_backup
Expand All @@ -31,6 +33,8 @@
when: cloud_service_provider == "aws"
- role: es-gcs-snapshot
when: cloud_service_provider == "gcloud"
- role: es-oci-snapshot
when: cloud_service_provider == "oci"
- es5-snapshot-purge
tags:
- log_es_backup
Expand Down
6 changes: 5 additions & 1 deletion ansible/inventory/env/group_vars/all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ content_service_blacklisted_resourcetype: ''
content_service_whitelisted_resourcetype: ''
content_service_whitelisted_mimetype: ''
content_service_blacklisted_mimetype: ''
sunbird_cloud_storage_urls: 'https://s3.ap-south-1.amazonaws.com/ekstep-public-{{ekstep_s3_env}}/,https://ekstep-public-{{ekstep_s3_env}}.s3-ap-south-1.amazonaws.com/,{{ upstream_url }}/'
sunbird_cloud_storage_urls: 'https://s3.ap-south-1.amazonaws.com/ekstep-public-{{ekstep_s3_env}}/,https://ekstep-public-{{ekstep_s3_env}}.s3-ap-south-1.amazonaws.com/,{{ upstream_url }}/,https://files.xyz.in/contentsbucket/,https://obj.xyz.in/contentsbucket/'
sunbird_email_max_recipients_limit: 100
sunbird_cassandra_consistency_level: local_quorum
sunbird_cassandra_replication_strategy: '{"class":"SimpleStrategy","replication_factor":"1"}'
Expand Down Expand Up @@ -566,3 +566,7 @@ plugin_upstream_url: "{{ upstream_url }}"
kp_schema_base_path: "{{ upstream_url }}/schemas/local"
# SB-31155 - Moved to the installation public container for now (same place where keycloaka and java artifacts are stored)
h5p_library_path: "https://sunbirdpublic.blob.core.windows.net/installation/h5p-standalone-1.3.4.zip"


## Added default values
cloud_public_storage_proxy: "{{cloud_storage_url}}"
13 changes: 13 additions & 0 deletions ansible/kp_upload-schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,16 @@
gcp_path: "schemas/local"
local_file_or_folder_path: "{{ source_name }}"
when: cloud_service_provider == "gcloud"

- name: upload batch of files to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_bucket_name: "{{ cloud_storage_content_bucketname }}"
local_file_or_folder_path: "{{ source_name }}"
oss_path: "schemas/local/"
when: cloud_service_provider == "oci"
4 changes: 2 additions & 2 deletions ansible/roles/cassandra/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
user_home: "/home/{{ ansible_ssh_user }}/"
cassandra_repo: 'deb http://www.apache.org/dist/cassandra/debian 311x main'
cassandra_repo_key: 'https://www.apache.org/dist/cassandra/KEYS'
cassandra_repo: 'deb https://debian.cassandra.apache.org 311x main'
cassandra_repo_key: 'https://downloads.apache.org/cassandra/KEYS'
resource_crunch: "yes" # make this value yes if heap size should be quarter of the Server Memory when you are deploying multiple databases, else make this value no to have 50% of the value as per the best practice of cassandra
cassandra_home_directory: "/var/lib/cassandra"
12 changes: 12 additions & 0 deletions ansible/roles/cert-templates/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,15 @@
gcp_path: ""
local_file_or_folder_path: "{{ cert_location }}/cert-templates/certUtilScripts/out"
when: cloud_service_provider == "gcloud"

- name: upload batch of files to oci oss
include_role:
name: oci-cloud-storage
apply:
environment:
OCI_CLI_AUTH: "instance_principal"
tasks_from: upload-folder.yml
vars:
oss_bucket_name: "{{ cert_service_container_name }}"
local_file_or_folder_path: "{{ cert_location }}/cert-templates/certUtilScripts/out"
when: cloud_service_provider == "oci"
19 changes: 19 additions & 0 deletions ansible/roles/es-oci-snapshot/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
snapshot_create_request_body: {
type: s3,
settings: {
bucket: "{{ cloud_storage_esbackup_bucketname }}",
path_style_access: "{{s3_path_style_access}}",
endpoint: "{{s3_storage_endpoint}}",
region: "{{s3_region}}",
base_path: "{{ cloud_storage_esbackup_foldername }}/{{ snapshot_base_path }}_{{ base_path_date }}",

}
}

# Override these values
es_snapshot_host: "localhost"
snapshot_base_path: "default"
es_backup_storage: "elasticsearch-snapshots"

cloud_storage_esbackup_bucketname: "{{ cloud_storage_management_bucketname }}"
cloud_storage_esbackup_foldername: "elasticsearch-snapshots"
Loading