@cchristelis the not allowed folder may be added on the robots.txt.
i tried to add the script on Dockerfile, but it seems that this repo is going to be further merged according to the need of the project.
so i made a shell script.

What do you think?

@boney-bun, so we should have this running when we don't change this image, but either way we need to add instructions of some sort to this repo to explain what we did here?

thanks for the feedback @cchristelis

I think you should also be adding an entry in the nginx configs to serve the robots file.

Yes, I agree with you.
previously, I added these lines on my local:

    location /robots.txt {
    	root $root_location
    }

but, the challenge is how to determine the root_location.
if we run the copy_robots.sh, we could obtain the root location.
but, i think the script can't be run from the conf file.

I tried running the original image locally, but failed to start.
that's why I think this repo may be need further modification based on the need of a project.

Also where are you running copy_robots.sh?

the copy_robots.sh suppose to be executed when the nginx has started so it will copy the robots.txt into the nginx's root location.

either way we need to add instructions of some sort to this repo to explain what we did here?

I agree. let me try to write some explanations and you can comment on it later.

have a look at how to copy files into containers during setup. You can determine the location.