Clone all Github Repos for a User

In this article I will describe a simple method you can use to clone all Github repos for a user. You may find it as surprising as I did that there is no built-in way to do this.

I recently came across an impressive collection of information security tools on Github. Very soon after that I found myself wanting to do further analysis on these repositories. I wanted to find a way to quickly clone all of a users repositories from Github.

Being the software giant that it is I was sure that Github would have a neat RESTful API. After a little digging I found the API endpoint that would list all repositories for a user. After that it was a simple matter of doing a little grepping to get a list of repository URLs, and then looping through them.

BASH command to list all (1000) repository URLs for a user

The default number of repositories returned by this endpoint is 30, so passing in 1000 will usually be high enough to grab all a user’s repositories.

curl -s https://api.github.com/users/<user_name>/repos?per_page=1000 |grep git_url |awk '{print $2}'| sed 's/"\(.*\)",/\1/'

After figuring out the basic command structure it was time to turn this into a useable script.

BASH script to clone all a user’s repositories

The script below will clone all repositories for the specified user into the current directory. Download it on Github: vke-code/download-all-repos.sh

#!/bin/bash
# Clone all github.com repositories for a specified user.

if [ $# -eq 0 ]
  then
    echo "Usage: $0 <user_name> "
    exit;
fi

USER=$1
# clone all repositories for user specifed
for repo in `curl -s https://api.github.com/users/$USER/repos?per_page=1000 |grep git_url |awk '{print $2}'| sed 's/"\(.*\)",/\1/'`;do
  git clone $repo;
done;

And that is all there is to it. With this script in hand you can easily clone all Github repos for a user. Nothing beats having a local archive in case the original is removed or deleted.

The great benefit to using this method is that it works for organizations as well as users. Just a small tweak to the API call and you can clone all repositories in an organization from Github.

Little exercises like this are always a great way to practice some scripting, or learn something new. If you liked this article you may also enjoy my article on brute force router hacking.

For more about the author, see the about page.

If you found this script helpful, please drop a comment below!

Leave a Reply

Your email address will not be published. Required fields are marked *