COLLECTED BY

Organization: Archive Team

Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.

History is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. With the original point of contention destroyed, the debates would fall to the wayside. Archive Team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. Our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over 100 volunteers stepping forward to acquire terabytes of user-created data to save for future generations.

The main site for Archive Team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs.

This collection contains the output of many Archive Team projects, both ongoing and completed. Thanks to the generous providing of disk space by the Internet Archive, multi-terabyte datasets can be made available, as well as in use by the Wayback Machine, providing a path back to lost websites and work.

Our collection has grown to the point of having sub-collections for the type of data we acquire. If you are seeking to browse the contents of these collections, the Wayback Machine is the best first stop. Otherwise, you are free to dig into the stacks to see what you may find.

The Archive Team Panic Downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures.

Collection: ArchiveBot: The Archive Team Crowdsourced Crawler

ArchiveBot is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive (or other archive sites).

To use ArchiveBot, drop by #archivebot on EFNet. To interact with ArchiveBot, you issue commands by typing it into the channel. Note you will need channel operator permissions in order to issue archiving jobs. The dashboard shows the sites being downloaded currently.

There is a dashboard running for the archivebot process at http://www.archivebot.com.

ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot.

TIMESTAMPS

The Wayback Machine - https://web.archive.org/web/20250726072208/https://github.blog/author/timrogers/

Skip to content

/ Blog

Try GitHub Copilot Attend GitHub Universe

AI & ML
- AI & ML
  Learn about artificial intelligence and machine learning across the GitHub ecosystem and the wider industry.
  - Generative AI
    Learn how to build with generative AI.
  - GitHub Copilot
    Change how you work with GitHub Copilot.
  - LLMs
    Everything developers need to know about LLMs.
  - Machine learning
    Machine learning tips, tricks, and best practices.
- How AI code generation works
  Explore the capabilities and benefits of AI code generation and how it can improve your developer experience.
  Learn more
Developer skills
- Developer skills
  Resources for developers to grow in their skills and careers.
  - Application development
    Insights and best practices for building apps.
  - Career growth
    Tips & tricks to grow as a professional developer.
  - GitHub
    Improve how you use GitHub at work.
  - GitHub Education
    Learn how to move into your first professional role.
  - Programming languages & frameworks
    Stay current on what’s new (or new again).
- Get started with GitHub documentation
  Learn how to start building, shipping, and maintaining software with GitHub.
  Learn more
Engineering
- Engineering
  Get an inside look at how we’re building the home for all developers.
  - Architecture & optimization
    Discover how we deliver a performant and highly available experience across the GitHub platform.
  - Engineering principles
    Explore best practices for building software at scale with a majority remote team.
  - Infrastructure
    Get a glimpse at the technology underlying the world’s leading AI-powered developer platform.
  - Platform security
    Learn how we build security into everything we do across the developer lifecycle.
  - User experience
    Find out what goes into making GitHub the home for all developers.
- How we use GitHub to be more productive, collaborative, and secure
  Our engineering and security teams do some incredible work. Let’s take a look at how we use GitHub to be more productive, build collaboratively, and shift security left.
  Learn more
Enterprise software
- Enterprise software
  Explore how to write, build, and deploy enterprise software at scale.
  - Automation
    Automating your way to faster and more secure ships.
  - CI/CD
    Guides on continuous integration and delivery.
  - Collaboration
    Tips, tools, and tricks to improve developer collaboration.
  - DevOps
    DevOps resources for enterprise engineering teams.
  - DevSecOps
    How to integrate security into the SDLC.
  - Governance & compliance
    Ensuring your builds stay clean.
- How enterprise engineering teams can successfully adopt AI
  Learn how to bring AI to your engineering teams and maximize the value that you get from it.
  Learn more
News & insights
- News & insights
  Keep up with what’s new and notable from inside GitHub.
  - Company news
    An inside look at news and product updates from GitHub.
  - Product
    The latest on GitHub’s platform, products, and tools.
  - Octoverse
    Insights into the state of open source on GitHub.
  - Policy
    The latest policy and regulatory changes in software.
  - Research
    Data-driven insights around the developer ecosystem.
  - The library
    Older news and updates from GitHub.
- Unlocking the power of unstructured data with RAG
  Learn how to use retrieval-augmented generation (RAG) to capture more insights.
  Learn more
Open Source
- Open Source
  Everything open source on GitHub.
  - Git
    The latest Git updates.
  - Maintainers
    Spotlighting open source maintainers.
  - Social impact
    How open source is driving positive change.
  - Gaming
    Explore open source games on GitHub.
- An introduction to innersource
  Organizations worldwide are incorporating open source methodologies into the way they build and ship their own software.
  Learn more
Security
- Security
  Stay up to date on everything security.
  - Application security
    Application security, explained.
  - Supply chain security
    Demystifying supply chain security.
  - Vulnerability research
    Updates from the GitHub Security Lab.
  - Web application security
    Helpful tips on securing web applications.
- The enterprise guide to AI-powered DevSecOps
  Learn about core challenges in DevSecOps, and how you can start addressing them with AI and automation.
  Learn more

Categories

AI & ML
- AI & ML
  Learn about artificial intelligence and machine learning across the GitHub ecosystem and the wider industry.
  - Generative AI
    Learn how to build with generative AI.
  - GitHub Copilot
    Change how you work with GitHub Copilot.
  - LLMs
    Everything developers need to know about LLMs.
  - Machine learning
    Machine learning tips, tricks, and best practices.
- How AI code generation works
  Explore the capabilities and benefits of AI code generation and how it can improve your developer experience.
  Learn more
Developer skills
- Developer skills
  Resources for developers to grow in their skills and careers.
  - Application development
    Insights and best practices for building apps.
  - Career growth
    Tips & tricks to grow as a professional developer.
  - GitHub
    Improve how you use GitHub at work.
  - GitHub Education
    Learn how to move into your first professional role.
  - Programming languages & frameworks
    Stay current on what’s new (or new again).
- Get started with GitHub documentation
  Learn how to start building, shipping, and maintaining software with GitHub.
  Learn more
Engineering
- Engineering
  Get an inside look at how we’re building the home for all developers.
  - Architecture & optimization
    Discover how we deliver a performant and highly available experience across the GitHub platform.
  - Engineering principles
    Explore best practices for building software at scale with a majority remote team.
  - Infrastructure
    Get a glimpse at the technology underlying the world’s leading AI-powered developer platform.
  - Platform security
    Learn how we build security into everything we do across the developer lifecycle.
  - User experience
    Find out what goes into making GitHub the home for all developers.
- How we use GitHub to be more productive, collaborative, and secure
  Our engineering and security teams do some incredible work. Let’s take a look at how we use GitHub to be more productive, build collaboratively, and shift security left.
  Learn more
Enterprise software
- Enterprise software
  Explore how to write, build, and deploy enterprise software at scale.
  - Automation
    Automating your way to faster and more secure ships.
  - CI/CD
    Guides on continuous integration and delivery.
  - Collaboration
    Tips, tools, and tricks to improve developer collaboration.
  - DevOps
    DevOps resources for enterprise engineering teams.
  - DevSecOps
    How to integrate security into the SDLC.
  - Governance & compliance
    Ensuring your builds stay clean.
- How enterprise engineering teams can successfully adopt AI
  Learn how to bring AI to your engineering teams and maximize the value that you get from it.
  Learn more
News & insights
- News & insights
  Keep up with what’s new and notable from inside GitHub.
  - Company news
    An inside look at news and product updates from GitHub.
  - Product
    The latest on GitHub’s platform, products, and tools.
  - Octoverse
    Insights into the state of open source on GitHub.
  - Policy
    The latest policy and regulatory changes in software.
  - Research
    Data-driven insights around the developer ecosystem.
  - The library
    Older news and updates from GitHub.
- Unlocking the power of unstructured data with RAG
  Learn how to use retrieval-augmented generation (RAG) to capture more insights.
  Learn more
Open Source
- Open Source
  Everything open source on GitHub.
  - Git
    The latest Git updates.
  - Maintainers
    Spotlighting open source maintainers.
  - Social impact
    How open source is driving positive change.
  - Gaming
    Explore open source games on GitHub.
- An introduction to innersource
  Organizations worldwide are incorporating open source methodologies into the way they build and ship their own software.
  Learn more
Security
- Security
  Stay up to date on everything security.
  - Application security
    Application security, explained.
  - Supply chain security
    Demystifying supply chain security.
  - Vulnerability research
    Updates from the GitHub Security Lab.
  - Web application security
    Helpful tips on securing web applications.
- The enterprise guide to AI-powered DevSecOps
  Learn about core challenges in DevSecOps, and how you can start addressing them with AI and automation.
  Learn more

Attend GitHub Universe Try GitHub Copilot

Tim Rogers

Staff Product Manager

Posts by this author

News & insights

From pair to peer programmer: Our vision for agentic workflows in GitHub Copilot

AI agents in GitHub Copilot don’t just assist developers but actively solve problems through multi-step reasoning and execution. Here’s what that means.

News & insights

The clock is ticking: Atlassian’s support for Bitbucket Server ends on February 15, 2024

Atlassian is ending support for its Server products—including Bitbucket Server—in February 2024. In this post, you’ll learn what that means for you, your options, and how you can move to GitHub.

Switching from Bitbucket Server and Bamboo to GitHub just got easier

Starting today, GitHub Enterprise Importer supports repository migrations from Bitbucket Server and Bitbucket Data Center, and GitHub Actions Importer offers CI/CD migrations from Bitbucket and Bamboo.

Cloud migration made easy: introducing GitHub Enterprise Importer

With GitHub Enterprise Importer, you can seamlessly move to GitHub Enterprise Cloud, bringing your code and collaboration history with you so your team doesn’t miss a beat.

Developer skills

To infinity and beyond: enabling the future of GitHub’s REST API with API versioning

We’re introducing calendar-based versioning for our REST API, so we can keep evolving our API, whilst still giving integrators a smooth migration path and plenty of time to update their integrations.