Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔥feat: Add real time console log stream #206

Merged
merged 2 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/sld-api-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ jobs:

- name: Build the Docker image with tag
working-directory: ./sld-api-backend
run: docker build . --file Dockerfile --tag ${{ secrets.DOCKER_USERNAME }}/sld-api:2.27.0
run: docker build . --file Dockerfile --tag ${{ secrets.DOCKER_USERNAME }}/sld-api:2.28.0

- name: Docker Push with tag
#if: github.event.pull_request.merged == true
run: docker push ${{ secrets.DOCKER_USERNAME }}/sld-api:2.27.0
run: docker push ${{ secrets.DOCKER_USERNAME }}/sld-api:2.28.0

- name: Build the Docker image
working-directory: ./sld-api-backend
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/sld-dashboard-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ jobs:

- name: Build the Docker image with tags
working-directory: ./sld-dashboard
run: docker build . --file Dockerfile --tag ${{ secrets.DOCKER_USERNAME }}/sld-dashboard:2.27.0
run: docker build . --file Dockerfile --tag ${{ secrets.DOCKER_USERNAME }}/sld-dashboard:2.28.0

- name: Docker Push with tags
#if: github.event.pull_request.merged == true
run: docker push ${{ secrets.DOCKER_USERNAME }}/sld-dashboard:2.27.0
run: docker push ${{ secrets.DOCKER_USERNAME }}/sld-dashboard:2.28.0
- name: Build the Docker image
working-directory: ./sld-dashboard
run: docker build . --file Dockerfile --tag ${{ secrets.DOCKER_USERNAME }}/sld-dashboard:latest
Expand Down
2 changes: 1 addition & 1 deletion play-with-sld/kubernetes/k8s/sld-api-backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
subdomain: primary
containers:
- name: api-backend
image: d10s0vsky/sld-api:2.27.0
image: d10s0vsky/sld-api:2.28.0
imagePullPolicy: Always
command: ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
ports:
Expand Down
2 changes: 1 addition & 1 deletion play-with-sld/kubernetes/k8s/sld-dashboard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
subdomain: primary
containers:
- name: sld-dashboard
image: d10s0vsky/sld-dashboard:2.27.0
image: d10s0vsky/sld-dashboard:2.28.0
env:
- name: PATH
value: "/home/sld/.asdf/shims:/home/sld/.asdf/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Expand Down
2 changes: 1 addition & 1 deletion play-with-sld/kubernetes/k8s/sld-worker-default.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
subdomain: primary
containers:
- name: stack-deploy-worker-default
image: d10s0vsky/sld-api:2.27.0
image: d10s0vsky/sld-api:2.28.0
imagePullPolicy: Always
env:
- name: TF_WARN_OUTPUT_ERRORS
Expand Down
2 changes: 1 addition & 1 deletion play-with-sld/kubernetes/k8s/sld-worker-squad1.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
subdomain: primary
containers:
- name: stack-deploy-worker-squad1
image: d10s0vsky/sld-api:2.27.0
image: d10s0vsky/sld-api:2.28.0
imagePullPolicy: Always
env:
- name: TF_WARN_OUTPUT_ERRORS
Expand Down
2 changes: 1 addition & 1 deletion play-with-sld/kubernetes/k8s/sld-worker-squad2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
subdomain: primary
containers:
- name: stack-deploy-worker-squad2
image: d10s0vsky/sld-api:2.27.0
image: d10s0vsky/sld-api:2.28.0
imagePullPolicy: Always
env:
- name: TF_WARN_OUTPUT_ERRORS
Expand Down
2 changes: 2 additions & 0 deletions sld-api-backend/src/worker/domain/services/command.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ def command(command: str, channel: str):
try:
output_lines = []
command_description = f"Executing command: {command}"
output_lines.append("-" * 80)
output_lines.append(command_description)
output_lines.append("-" * 80)
redis_client.publish(channel, "-" * 80)
redis_client.publish(channel, command_description)
redis_client.publish(channel, "-" * 80)
Expand Down
15 changes: 9 additions & 6 deletions sld-api-backend/src/worker/providers/hashicorp/actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,18 @@ def execute_terraform_command(self, action: str) -> dict:
plan_command = f"/tmp/{self.version}/terraform plan -input=false -refresh -no-color -var-file={variables_files} -out={self.name}.tfplan"
apply_command = f"/tmp/{self.version}/terraform apply -input=false -auto-approve -no-color {self.name}.tfplan"
destroy_command = f"/tmp/{self.version}/terraform destroy -input=false -auto-approve -no-color -var-file={variables_files}"

output = []
if action == "plan":
result, output = command(init_command, channel=channel)
result, output = self.subprocess_handler(plan_command, channel=channel)
result, output_init = command(init_command, channel=channel)
result, output_plan = self.subprocess_handler(plan_command, channel=channel)
output = output_init + output_plan
if action == "apply":
result, output = self.subprocess_handler(apply_command, channel=channel)
result, output_apply = self.subprocess_handler(apply_command, channel=channel)
output = output + output_apply
elif action == "destroy":
result, output = command(init_command, channel=channel)
result, output = self.subprocess_handler(destroy_command, channel=channel)
result, output_init = command(init_command, channel=channel)
result, output_destroy = self.subprocess_handler(destroy_command, channel=channel)
output = output_init + output_destroy

unsecret(self.stack_name, self.environment, self.squad, self.name, self.secreto)

Expand Down
4 changes: 2 additions & 2 deletions sld-dashboard/app/base/static/assets/css/volt.css
Original file line number Diff line number Diff line change
Expand Up @@ -41830,8 +41830,8 @@ pre {

.beta-icon {
font-weight: bold;
font-size: 18px;
background-color: #d9534f; /* Color de fondo para el ícono */
color: var(--bs-dark);
font-size: 25px;
padding: 5px 10px;
border-radius: 4px;
margin-right: 5px;
Expand Down
2 changes: 1 addition & 1 deletion sld-dashboard/app/home/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def deploy_stream(task_id):
@login_required
def stream(task_id):
def generate():
pubsub = s.pubsub(ignore_subscribe_messages=True)
pubsub = s.pubsub(ignore_subscribe_messages=False)
pubsub.subscribe(f'{task_id}')
for message in pubsub.listen():
logging.info(message)
Expand Down
30 changes: 26 additions & 4 deletions sld-dashboard/app/home/templates/deploy-stream.html
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@
<main class="content">
{% include 'includes/navigation.html' %}
<div class="beta-version">
<span class="beta-icon">BETA</span>
<span class="beta-icon">Console Log -></span>
<span class="fab fa-docker fa-2x mr-0"></span>
<p>This is a beta version of the feature. Feedback is welcome!</p>
</div>
<div class="preloader bg-soft flex-column justify-content-center align-items-center">
Expand Down Expand Up @@ -72,15 +73,36 @@ <h2>Deployment Output:</h2>
eventSource.onmessage = function(e) {
const sseDataElement = document.getElementById('sse-data');

if (e.data !== '1') {
if (e.data !== '1' && e.data !== 'null') {
// Handle normal data, excluding 'null'
sseDataElement.textContent += e.data + '\n';
// Delay scroll adjustment to ensure the browser has processed the new content
setTimeout(() => {
const lastLine = document.createElement("div");
sseDataElement.appendChild(lastLine);
lastLine.scrollIntoView({ behavior: "smooth", block: "end" });
}, 30);
}
} else if (typeof e.data === 'string') {
// Handle the case where '1' is received - fetch additional data
fetch('/output/' + taskId)
.then(response => response.json()) // Parse the JSON response
.then(dataArray => {
if (dataArray && dataArray.length > 0) {
// Iterate over each string in the array and append it, excluding 'null'
dataArray.forEach(item => {
if (item && item !== 'null') {
sseDataElement.textContent += item + '\n';
}
});
} else {
sseDataElement.textContent = 'No additional task data available.';
}
})
.catch(error => {
console.error('Error fetching additional task data:', error);
sseDataElement.textContent = 'Error fetching data.';
});
}
};
</script>

{% endblock javascripts %}
6 changes: 3 additions & 3 deletions sld-dashboard/app/home/templates/deploys-list.html
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ <h2 class="h4">All Deploys</h2>
<!-- Output-->
<!-- SSE stream -->
<span class="icon icon-sm">
<a title="Output Stream" class="dropdown-item" href="{{ url_for('.deploy_stream',task_id=deploy.task_id)}}">
<a title="Console Log Stream" class="dropdown-item" href="{{ url_for('.deploy_stream',task_id=deploy.task_id)}}">
<span class="fab fa-searchengin mr-0"></span>
</a>
</span>
Expand Down Expand Up @@ -531,9 +531,9 @@ <h5 class="modal-title" id="DeleteModalLongTitle-{{deploy.id}}">
// Función auxiliar para determinar la clase CSS según el contenido de la línea
function getClassForLine(line) {
if (/^\s*\+/.test(line)) return 'plus';
if (/^\s*-/.test(line)) return 'minus';
if (/^\s*-(?!\-)/.test(line)) return 'minus'; // Modificado para un solo guion '-'
if (/^\s*~/.test(line)) return 'tilde';
return 'others';
return 'others';
}
// Asignar el evento click a los botones para abrir el modal
document.querySelectorAll('.output-button').forEach(button => {
Expand Down
Loading