scrape permissions
git-scraping is awesome. It let's you re-use CI as a pattern to scrape data from the internet and store it publicly in your repository. Here's an example:
name: scrape
on:
schedule:
- cron: '0 * * * *' # hourly
workflow_dispatch:
permissions:
contents: write # needed to push the data back
jobs:
scrape:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: curl -s https://example.com/data.json > data.json
- run: |
git config user.name "Automated"
git config user.email "actions@users.noreply.github.com"
git add -A
git commit -m "Latest data: $(date -u)" || exit 0
git push
The best part of this is that you only need Git. Everyone already has access to Git. And it also means that everyone can have access to important analytics. No need to add another tool! Not to mention the fact that integrating with a GitHub account is dead simple compared to integrating with any analytics tool.
permissions
One small tip if you do this at a company: don't forget about the permission flag. At some point security will, rightfully, be hardened. And you typically don't want CI jobs to make extra merge requests to your repo. So you'll need to do this:
name: scrape
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
permissions:
contents: write # this part!
jobs:
scrape:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: curl -s https://example.com/data.json > data.json
- run: |
git config user.name "Automated"
git config user.email "actions@users.noreply.github.com"
git add -A
git commit -m "Latest data: $(date -u)" || exit 0
git push
Without that permission, jobs may fail in the future.