veganism.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
Veganism Social is a welcoming space on the internet for vegans to connect and engage with the broader decentralized social media community.

Administered by:

Server stats:

293
active users

#scrapy

0 posts0 participants0 posts today
Renne Rocha<p>This week I wrote how to use "CrawlSpider" to use a declarative format to follow links during a web scraping project with Scrapy. </p><p>However, in a past project, I had the need to extend this functionality a bit, defining dynamic rules (based on user input).</p><p>So as a continuation of my previous post, I wrote a new one explaining a little about how this solution was made.</p><p><a href="https://rennerocha.com/posts/dynamic-rules-for-following-links-declaratively-with-scrapy/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">rennerocha.com/posts/dynamic-r</span><span class="invisible">ules-for-following-links-declaratively-with-scrapy/</span></a></p><p><a href="https://chaos.social/tags/webscraping" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webscraping</span></a> <a href="https://chaos.social/tags/scrapy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scrapy</span></a> <a href="https://chaos.social/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a></p>
Renne Rocha<p>Scrapy has a generic Spider class (`CrawlSpider`) that helps extracting and following links in a more declarative way (using a set of "rules"). I noticed in projects that I worked before, that this not used that much, but I found it very helpful and make the code very readable.</p><p>I wrote something about it:<br><a href="https://rennerocha.com/posts/following-links-declaratively-with-scrapy/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">rennerocha.com/posts/following</span><span class="invisible">-links-declaratively-with-scrapy/</span></a></p><p><a href="https://chaos.social/tags/webscraping" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>webscraping</span></a> <a href="https://chaos.social/tags/scrapy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>scrapy</span></a> <a href="https://chaos.social/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a></p>
Makaze<p><a href="https://github.com/Makaze/csschooser" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">github.com/Makaze/csschooser</span><span class="invisible"></span></a></p><p>Interactive CLI CSS Chooser for making BeautifulSoup and Scrapy scripts.</p><p>CS50P certificate achieved in &lt;1 week! I'm glad I did it; I've needed to make something for quite some time. <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> <a href="https://hachyderm.io/tags/CS50" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CS50</span></a> <a href="https://hachyderm.io/tags/CS50p" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CS50p</span></a> <a href="https://hachyderm.io/tags/CSS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CSS</span></a> <a href="https://hachyderm.io/tags/BeautifulSoup" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BeautifulSoup</span></a> <a href="https://hachyderm.io/tags/Scrapy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Scrapy</span></a></p>