Skip to content

Commit

Permalink
Expose list of pages to scrape with JavaScript
Browse files Browse the repository at this point in the history
#15639

This exposes a dynamically generated text file with a list of URLs that
should be generated with JS enabled by the static site generation
process. This can be read by the script added in d8b3c9f to determine
what URLs it should scrape.

It currently only contains a single line for the one page we want to
scrape after JS has been run. In the future it should be relatively
easy to add new URLs to this list including lists of URLs
programmatically generated.

Now that we have a script to do the JS scraping and a way of generating
a list of URLs for this process, our next step is to add this to the
build process.
  • Loading branch information
henare committed Jan 18, 2018
1 parent a3f7702 commit 7fd0d10
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions app.rb
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@
erb :needed
end

get '/javascript_pages_to_scrape.txt' do
headers 'Content-Type' => 'text/plain'
body url('/needed.html')
end

get '/*.css' do |filename|
scss :"sass/#{filename}"
end
Expand Down

0 comments on commit 7fd0d10

Please sign in to comment.