René's URL Explorer Experiment

Title: Added new git-repo-scanner options (rate limit, activity filter) by rfelber · Pull Request #327 · secureCodeBox/secureCodeBox · GitHub

Open Graph Title: Added new git-repo-scanner options (rate limit, activity filter) by rfelber · Pull Request #327 · secureCodeBox/secureCodeBox

X Title: Added new git-repo-scanner options (rate limit, activity filter) by rfelber · Pull Request #327 · secureCodeBox/secureCodeBox

Description: Description Implements a new rate limit configuration Implements #325 This PR if applied adds a new rate limit configuration to the git-repo-scanner to obey the GitHub and GitLeaks server rate limits. By default this setting is true. Example Works for GitHub and Gitlab: --obey-rate-limit: True to obey the rate limit of the GitHub server (default), otherwise False For unauthenticated requests, the rate limit allows for up to 60 requests per hour. Unauthenticated requests are associated with the originating IP address, and not the user making requests. With --obey-rate-limit=True the scanner will hit the GitHub API every 60 seconds at max (3600s / 60 requests per hour = 60s seconds delay between two request). OAuth applications authorized by a user share the same quota of 5,000 requests per hour when they authenticate with different tokens owned by the same user. With --obey-rate-limit=True the scanner will hit the GitHub API every 0,7 seconds at max (3600s / 5000 requests per hour = 0,7s seconds delay between two request). Implements a new git repository filter based on latest activity Implements #320 Example Works for GitHub and Gitlab: --activity-since-duration: Return git repo findings with repo activity (e.g. commits) more recent than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'. --activity-until-duration: Return git repo findings with repo activity (e.g. commits) older than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'. Testing If you want to test this PR please reference the following docker image: docker pull securecodebox/scanner-git-repo-scanner:git-repo-rate-limit image: # image.repository -- Container Image to run the scan repository: docker.io/securecodebox/scanner-git-repo-scanner # image.tag -- defaults to the charts version tag: git-repo-rate-limit Example HelmChart Install You can use the following command to install and test this PR in your environment. For more general informations please check the scanner documentation : https://docs.securecodebox.io/docs/scanners/git-repo-scanner # Optional: Add the secureCodeBox Helm Repo helm repo add secureCodeBox https://charts.securecodebox.io # Install HelmChart (use -n to configure another namespace) helm upgrade --install git-repo-scanner secureCodeBox/git-repo-scanner \ --set="image.repository=docker.io/securecodebox/scanner-git-repo-scanner" \ --set="image.tag=git-repo-rate-limit" Checklist Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests. Make sure npm test runs for the whole project. Cleanup the code to make codeclimate happy

Open Graph Description: Description Implements a new rate limit configuration Implements #325 This PR if applied adds a new rate limit configuration to the git-repo-scanner to obey the GitHub and GitLeaks server rate limi...

X Description: Description Implements a new rate limit configuration Implements #325 This PR if applied adds a new rate limit configuration to the git-repo-scanner to obey the GitHub and GitLeaks server rate limi...

Opengraph URL: https://github.com/secureCodeBox/secureCodeBox/pull/327

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width