| Index | index by Group | index by Distribution | index by Vendor | index by creation date | index by Name | Mirrors | Help | Search |
| Name: python-Scrapy-doc | Distribution: openSUSE Tumbleweed |
| Version: 2.17.0+git7 | Vendor: openSUSE |
| Release: 1.1 | Build date: Tue Jul 14 16:26:35 2026 |
| Group: Unspecified | Build host: reproducible |
| Size: 0 | Source RPM: python-Scrapy-2.17.0+git7-1.1.src.rpm |
| Packager: http://bugs.opensuse.org | |
| Url: https://scrapy.org | |
| Summary: Documentation for python-Scrapy | |
Provides documentation for python-Scrapy.
BSD-3-Clause
* Tue Jul 14 2026 Nico Krapp <nico.krapp@suse.com>
- Update to version 2.17.0+git7:
* Assorted docs fixes, part 2. (#7725)
* Solve the feed Path issue (#7674)
* Use external mitmproxy. (#7720)
* Fix silent data loss in CsvItemExporter (closes #4002) (#7651)
* More granular untyped defs checking in tests. (#7712)
* Assorted docs fixes, part 1. (#7710)
* Update tools (#7724)
* Bump version: 2.16.0 → 2.17.0
* Merge commit from fork
* Release notes for 2.17.0 (#7723)
* LxmlLinkExtractor: add deny_attrs and deny_tags (#7679)
* fix-utf16-response-test-on-big-endian-systems (#7508)
* Improve test coverage for crawler.py (#7682)
* Clarify/cleanup Selector.type (#7704)
* Keep Item fields in definition order (#7694)
* Assorted docstring fixes (#7698)
* Fix _get_tag_name() crash for non-string elem.tag (#7686) (#7687)
* Fix genspider --editor (#7683)
* Cleanup cookie handling in request_to_curl() (#7684)
* Fix to LocalCache with limit=0 still stores items rather than disabling the cache (#7663)
* Update on Request objects (#7286)
* Solve timing issues with HTTP cache tests? (#7692)
* Fix request_to_curl() corrupting dict cookies with bytes keys/values (#7675)
* Close various garbage collectible resources (mostly in tests) (#7644)
* Undeprecated the basic FormRequest API (#7671)
* Improve test coverage for shell/ (#7680)
* Improve the item pipeline docs (#7676)
* Add a documentation page about security (#7678)
* Improve test coverage for contracts (#7677)
* Improve test coverage for http/ (#7672)
* Fix FTPDownloadHandler not closing FTP connection after download (#7667)
* Make DOWNLOADER_CLIENT_TLS_CIPHERS=None enable Twisted defaults (#7665)
* Document “logging settings” as special settings (#7668)
* Improve test coverage for spider middlewares (#7664)
* Improve test coverage for downloader middlewares (#7655)
* Improve test coverage for settings/ (#7654)
* Add content-based image filtering example (#4954)
* Fix cell-var-from-loop bug in _send_catch_log_deferred (#7649)
* Document scrapy-lint, remove start_url check (#7627)
* Improve test coverage for responsetypes.py (#7646)
* Do not ignore CrawlerProcess settings (#7647)
* Add test coverage for scrapy.utils.decorators (#7645)
* Refactor and improve catching warnings in tests. (#7643)
* Improve coverage resolver (#7642)
* Improve coverage statscollectors (#7641)
* Improve test coverage for pqueues.py (#7640)
* Complete test coverage for linkextractors (#7639)
* Address warnings in tests. (#7637)
* Deprecate ScrapyCommand.help() (#7633)
* tests: integration coverage for memusage extension (#7017)
* Fix Proxy-Authorization handling in BaseStreamingDownloadHandler (#7630)
* Fix rel_has_nofollow to be case-insensitive per HTML spec (#7632)
* Fix image store ACL settings for S3 and GCS (#7614)
* Test SignalManager.disconnect_all() (#7625)
* Fix request_to_curl handling of verbose cookies dictionaries (#7603)
* Fix deprecation warnings with pytest 9.1. (#7621)
* Fix deprecation warnings with pyOpenSSL 26.3.0. (#7619)
* Fix strip_url() removing default port from password. (#7605)
* Assorted test fixes (#7616)
* Complete coverage for addons.py (#7612)
* Link.__eq__: return NotImplemented instead of raising NotImplementedError (#7611)
* Add coverage for DummySpiderLoader.find_by_request() (#7608)
* Add coverage for Item.__delitem__() (#7610)
* Implement HTTP Auth settings and request metadata keys (#7590)
* Document the SPIDER_MODULES hack to lower startup time and memory usage (#7600)
* Document cb_kwargs/meta deep-copy when JOBDIR is set (#7573)
* Clarify the first-class-citizen nature of asyncio support (#7599)
* Small docs fixes. (#7598)
* Remove set_asyncio_event_loop call from Shell._schedule (#7594)
* Validate reversed telnet console port ranges (#7593)
* tox.ini: pinned, typing → min, mypy (#7595)
* Prevent scrapy shell from starting spider requests (#7557)
* Fix the check for sync ITEM_PROCESSOR methods. (#7589)
* Update default_settings, deprecate CRAWLSPIDER_FOLLOW_LINKS. (#7592)
* Replace _SettingsKeyT with str. (#7586)
* Assorted test fixes (#7585)
* Add support for HTTP/2 and for SOCKS proxies to HttpxDownloadHandler, improve handler docs (#7575)
* Allow configuring the log level of the retry give-up message (#7567)
* Add settings for TLS min/max version as a replacement for the TLS method (#6546)
* docs: switch `scrapy.Item` examples to dataclasses (#7513)
* Work around coverage slowness on Python 3.14. (#7574)
* Support sending requests with "unsafe" URLs (#7473)
* Close GCS feed temp files after upload (#7546)
* fix open_in_browser() logic issue plus add new tests (#7506)
* Refactor test_crawl.py and test_crawler.py. (#7566)
* Convert tests that fail with testfixtures 12.0.0 (#7545)
* fix typos (#7564)
* DOC -> Add missing documentation for CloseSpider & CoreStats (#7421)
* Bump version: 2.15.2 → 2.16.0
* Release notes for Scrapy 2.16.0. (#7536)
* Disable mypy allow_any_generics. (#7532)
* Add BaseStreamingDownloadHandler and migrate HttpxDownloadHandler to it. (#7524)
* Remove support for sync process_spider_output(). (#7504)
* Fix certificate issuer verification on new Twisted. (#7520)
* Document TLS method setting in avoiding bans guide (#7518)
* Add CITATION.cff (#7519)
* Make more of the internal handler helpers private. (#7510)
* Add Python 3.14 to CI. (#6604)
- switch scm service to git offset
* Tue May 12 2026 mcepl@cepl.eu
- Update to version 2.15.2+git.1778596491.3b34ab88c:
* docs: document daily log file rotation (#7501)
* Update the TLS code to Twisted 26.4.0 (#7347)
* Cleanup/clarifications of proxy support and improvements for proxy tests (#7496)
* sphinx-scrapy: 0.8.5 → 0.8.6 (#7507)
* Pin pyOpenSSL to the versions allowing mutable contexts. (#7494)
* Enable mypy warn_return_any. (#7492)
* Silence deprecation warnings in TestFormRequest. (#7491)
* chore: simplify some code, get rid of nested fns where it's makes sence + pylint (#7401)
* Deprecate FormRequest in favor of form2request (#6438)
* Drop start_requests() support. (#7490)
* Tue May 12 2026 Matej Cepl <mcepl@cepl.eu>
- Add twisted-26_0.patch to make package compatible with Twisted
26.0+
* Sun May 03 2026 Dirk Müller <dmueller@suse.com>
- update to 2.15.2:
* Fixed links in https://docs.scrapy.org/llms.txt
(:issue:`7467`)
* Sharing of the SSL context between multiple connections,
introduced in 2.15.0
* Thu Apr 23 2026 Steve Kowalik <steven.kowalik@suse.com>
- Update to 2.15.0:
* Experimental support for running without a Twisted reactor
* Experimental httpx-based download handler
* Values from the Referrer-Policy header of HTTP responses are no longer
executed as Python callables.
* In line with the standard, 301 redirects of POST requests are converted
into GET requests
* Deprecate maybeDeferred_coro()
* Pass the spider arg to custom stat collectors {open,close}_spider()
* More coroutine-based replacements for Deferred-based APIs
* The default priority queue is now DownloaderAwarePriorityQueue
* Dropped support for Python 3.9 and PyPy 3.10
* Fix for the CVE-2025-6176 security issue: improved protection against
decompression bombs in HttpCompressionMiddleware for responses compressed
using the br and deflate methods. Requires brotli >= 1.2.0.
- Dropped patches:
* remove-hoverxref.patch
* no-dark-mode.patch
* CVE-2025-6176.patch
- Clean up BuildRequires, remove duplication from Requires.
* Mon Nov 17 2025 Daniel Garcia <daniel.garcia@suse.com>
- Update CVE-2025-6176.patch to reflect the latest changes upstream to
the patch.
- Remove the CVE-2025-6176-testfile-bomb-br-64GiB.bin source, it's not
needed anymore.
gh#scrapy/scrapy#7134, bsc#1252945, CVE-2025-6176)
* Wed Nov 12 2025 Daniel Garcia <daniel.garcia@suse.com>
- Use libalternatives
- Use multibuild to run tests in a subpackage
- add upstream patch CVE-2025-6176.patch to mitigate brotli and
deflate decompression bombs DoS.
This patch adds a new bin test file that was added as a new source
as CVE-2025-6176-testfile-bomb-br-64GiB.bin
gh#scrapy/scrapy#7134, bsc#1252945, CVE-2025-6176)
* Thu Jul 31 2025 Steve Kowalik <steven.kowalik@suse.com>
- Update to 2.13.3:
* Changed the values for DOWNLOAD_DELAY (from 0 to 1) and
CONCURRENT_REQUESTS_PER_DOMAIN (from 8 to 1) in the default project
template.
* Fixed several bugs in the engine initialization and exception handling
logic.
* Allowed running tests with Twisted 25.5.0+ again and fixed test failures
with lxml 6.0.0.
* Give callback requests precedence over start requests when priority
values are the same.
* The asyncio reactor is now enabled by default
* Replaced start_requests() (sync) with start() (async) and changed how it
is iterated.
* Added the allow_offsite request meta key
* Spider middlewares that don't support asynchronous spider output are
deprecated
* Added a base class for universal spider middlewares
- Add patch remove-hoverxref.patch:
* Do not use deprecated sphinx-hoverxref extension.
- Add patch no-dark-mode.patch:
* Do not use unavailable sphinx-rtd-dark-mode extension.
* Thu Mar 27 2025 Steve Kowalik <steven.kowalik@suse.com>
- Normalize metadata directory name.
* Tue Dec 03 2024 Steve Kowalik <steven.kowalik@suse.com>
- Update to 2.12.0:
* Dropped support for Python 3.8, added support for Python 3.13
* start_requests can now yield items
* Added scrapy.http.JsonResponse
* Added the CLOSESPIDER_PAGECOUNT_NO_ITEM setting
* Thu Jul 11 2024 Dirk Müller <dmueller@suse.com>
- update to 2.11.2 (bsc#1224474, CVE-2024-1968):
* Redirects to non-HTTP protocols are no longer followed.
Please, see the 23j4-mw76-5v7h security advisory for more
information. (:issue:`457`)
* The Authorization header is now dropped on redirects to a
different scheme (http:// or https://) or port, even if the
domain is the same. Please, see the 4qqq-9vqf-3h3f security
advisory for more information.
* When using system proxy settings that are different for
http:// and https://, redirects to a different URL scheme
will now also trigger the corresponding change in proxy
settings for the redirected request. Please, see the
jm3v-qxmh-hxwv security advisory for more information.
(:issue:`767`)
* :attr:`Spider.allowed_domains
<scrapy.Spider.allowed_domains>` is now enforced for all
requests, and not only requests from spider callbacks.
* :func:`~scrapy.utils.iterators.xmliter_lxml` no longer
resolves XML entities.
* defusedxml is now used to make
:class:`scrapy.http.request.rpc.XmlRpcRequest` more secure.
* Restored support for brotlipy_, which had been dropped in
Scrapy 2.11.1 in favor of brotli. (:issue:`6261`) Note
brotlipy is deprecated, both in Scrapy and upstream. Use
brotli instead if you can.
* Make :setting:`METAREFRESH_IGNORE_TAGS` ["noscript"] by
default. This prevents :class:`~scrapy.downloadermiddlewares.
redirect.MetaRefreshMiddleware` from following redirects that
would not be followed by web browsers with JavaScript
enabled.
* During :ref:`feed export <topics-feed-exports>`, do not close
the underlying file from :ref:`built-in post-processing
plugins <builtin-plugins>`.
* :class:`LinkExtractor
<scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor>` now
properly applies the unique and canonicalize parameters.
* Do not initialize the scheduler disk queue if
:setting:`JOBDIR` is an empty string.
* Fix :attr:`Spider.logger <scrapy.Spider.logger>` not logging
custom extra information.
* robots.txt files with a non-UTF-8 encoding no longer prevent
parsing the UTF-8-compatible (e.g. ASCII) parts of the
document.
* :meth:`scrapy.http.cookies.WrappedRequest.get_header` no
longer raises an exception if default is None.
:func:`scrapy.utils.response.get_base_url` to determine the
base URL of a given :class:`~scrapy.http.Response`.
* :class:`~scrapy.selector.Selector` now uses
:func:`scrapy.utils.response.get_base_url` to determine the
base URL of a given :class:`~scrapy.http.Response`.
(:issue:`6265`)
* The :meth:`media_to_download` method of :ref:`media pipelines
<topics-media-pipeline>` now logs exceptions before stripping
them.
* When passing a callback to the :command:`parse` command,
build the callback callable with the right signature.
* Add a FAQ entry about :ref:`creating blank requests <faq-
blank-request>`.
* Document that :attr:`scrapy.selector.Selector.type` can be
"json".
* Make builds reproducible.
* Packaging and test fixes
* Mon Mar 25 2024 Dirk Müller <dmueller@suse.com>
- update to 2.11.1 (bsc#1220514, CVE-2024-1892, bsc#1221986):
* Addressed `ReDoS vulnerabilities` (bsc#1220514, CVE-2024-1892)
- ``scrapy.utils.iterators.xmliter`` is now deprecated in favor of
:func:`~scrapy.utils.iterators.xmliter_lxml`, which
:class:`~scrapy.spiders.XMLFeedSpider` now uses.
To minimize the impact of this change on existing code,
:func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
the node namespace with a prefix in the node name, and big files with
highly nested trees when using libxml2 2.7+.
- Fixed regular expressions in the implementation of the
:func:`~scrapy.utils.response.open_in_browser` function.
.. _ReDoS vulnerabilities: https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
* :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
advisory`_ for more information. (bsc#1221986)
.. _7j7m-v7m3-jqm7 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7
* Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, the
deprecated ``scrapy.downloadermiddlewares.decompression`` module has been
removed.
* The ``Authorization`` header is now dropped on redirects to a different
domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
information.
* The OS signal handling code was refactored to no longer use private Twisted
functions. (:issue:`6024`, :issue:`6064`, :issue:`6112`)
* Improved documentation for :class:`~scrapy.crawler.Crawler` initialization
changes made in the 2.11.0 release. (:issue:`6057`, :issue:`6147`)
* Extended documentation for :attr:`Request.meta <scrapy.http.Request.meta>`.
* Fixed the :reqmeta:`dont_merge_cookies` documentation. (:issue:`5936`,
* Added a link to Zyte's export guides to the :ref:`feed exports
* Added a missing note about backward-incompatible changes in
:class:`~scrapy.exporters.PythonItemExporter` to the 2.11.0 release notes.
* Added a missing note about removing the deprecated
``scrapy.utils.boto.is_botocore()`` function to the 2.8.0 release notes.
* Other documentation improvements. (:issue:`6128`, :issue:`6144`,
:issue:`6163`, :issue:`6190`, :issue:`6192`)
- drop twisted-23.8.0-compat.patch (upstream)
* Wed Jan 10 2024 Daniel Garcia <daniel.garcia@suse.com>
- Add patch twisted-23.8.0-compat.patch gh#scrapy/scrapy#6064
- Update to 2.11.0:
- Spiders can now modify settings in their from_crawler methods,
e.g. based on spider arguments.
- Periodic logging of stats.
- Bug fixes.
- 2.10.0:
- Added Python 3.12 support, dropped Python 3.7 support.
- The new add-ons framework simplifies configuring 3rd-party
components that support it.
- Exceptions to retry can now be configured.
- Many fixes and improvements for feed exports.
- 2.9.0:
- Per-domain download settings.
- Compatibility with new cryptography and new parsel.
- JMESPath selectors from the new parsel.
- Bug fixes.
- 2.8.0:
- This is a maintenance release, with minor features, bug fixes, and
cleanups.
/usr/share/doc/packages/python-Scrapy-doc /usr/share/doc/packages/python-Scrapy-doc/html
Generated by rpm2html 1.8.1
Fabrice Bellet, Sun Aug 2 03:14:27 2026