Discover / Data & Research
Scrapy
by scrapyPython
The veteran Python scraping framework for structured data extraction at scale.
Repositorystable
Maturity: stable because 16y old, 2.17.0 released 27d ago. Derived from release and commit history, not a rating.
- Stars
- 64k
- Forks
- 12k
- Downloads / mo
- 3.7M
- Last commit
- 2026-08-01
- License
- BSD-3-Clause
- Open issues
- 571
Market and trust evidence
Edition not yet matchedNo exact skills.sh identity match is available for this repository. Repository adoption and freshness remain visible above; install momentum is not inferred.
Trust analysis is a screening signal, not a security warranty. Read the ranking and trust methodology.
In practice
Written by AI from this repository’s README · high confidenceDevelopers need a robust framework to extract structured data from websites efficiently.
Use it when
When you need a cross-platform, high-level web scraping framework built in Python.
Not the right pick when
When your environment is older than Python 3.10, as it requires Python 3.10+.
Capabilities
- Extract structured data from websites
- Cross-platform support
- HTTP/2 and SOCKS proxy support
Requirements
- Python 3.10+
Cost: Free and open source
Install
Derived from the published package name in the repository, not from a model.
Video walkthroughs
Scrapy Tutorial for Beginners: Web Scraping with Python 2026
Python Scrapy Tutorial #1 - Introduction, Setup & Project Structure
Third-party YouTube uploads matched to this tool by title, channel and repository name on 2026-08-03. Not made, reviewed or endorsed by SkillPilot. View counts and publish months are as of the match date and the month is approximate. Nothing loads from YouTube until you press play.
What the repository ships
Has testsHas docsSecurity policyCI configured
Detected from the actual files in the repository root.
Latest release 2.17.0
Published 2026-07-07
- Security bug fixes
- HTTP/2 and SOCKS proxy support for HttpxDownloadHandler
- Improved settings for changing allowed TLS versions
Tags
README
|logo|
.. |logo| image:: https://raw.githubusercontent.com/scrapy/scrapy/master/docs/_static/logo.svg
:target: https://scrapy.org
:alt: Scrapy
:width: 480px
|version| |python_version| |tests| |coverage| |conda| |deepwiki|
.. |version| image:: https://img.shields.io/pypi/v/Scrapy.svg
:target: https://pypi.org/pypi/Scrapy
:alt: PyPI Version
.. |python_version| image:: https://img.shields.io/pypi/pyversions/Scrapy.svg
:target: https://pypi.org/pypi/Scrapy
:alt: Supported Python Versions
.. |tests| image:: https://img.shields.io/github/check-runs/scrapy/scrapy/master?label=tests
:target: https://github.com/scrapy/scrapy/actions?query=branch%3Amaster
:alt: Tests
.. |coverage| image:: https://img.shields.io/codecov/c/github/scrapy/scrapy/master.svg
:target: https://codecov.io/github/scrapy/scrapy?branch=master
:alt: Coverage report
.. |conda| image:: https://anaconda.org/conda-forge/scrapy/badges/version.svg
:target: https://anaconda.org/conda-forge/scrapy
:alt: Conda Version
.. |deepwiki| image:: https://deepwiki.com/badge.svg
:target: https://deepwiki.com/scrapy/scrapy
:alt: Ask DeepWiki
Scrapy_ is a web scraping framework to extract structured data from websites.
It is cross-platform, and requires Python 3.10+. It is maintained by Zyte_
(formerly Scrapinghub) and many other contributors_.
.. _many other contributors: https://github.com/scrapy/scrapy/graphs/contributors
.. _Scrapy: https://scrapy.org/
.. _Zyte: https://www.zyte.com/
Install with:
.. code:: bash
pip install scrapy
And follow the documentation_ to learn how to use it.
.. _documentation: https://docs.scrapy.org/en/latest/
If you wish to contribute, see Contributing_.
.. _Contributing: https://docs.scrapy.org/en/master/contributing.html