Prestashop is one of the most popular e-commerce platforms in the world, prized for the flexibility and ease with which it allows people to manage online stores. However, those who run a Prestashop-based store often encounter problems with indexing and budget crawling optimization, especially when using complex modules such as ps_facetedsearch.
In fact, the Appearance Search module(ps_facetedsearch), which allows users to filter products based on various attributes (price, color, size, etc.), can generate a high number of similar or duplicate URLs. This phenomenon not only causes difficulty for search engines in determining which page is relevant, but also negatively impacts your crawling budget, i.e., the time and resources Google devotes to crawling your site's pages.
In this article we will explore how to effectively address these problems, highlighting the importance of the ps_facetedsearch module, providing practical solutions to optimize its use and significantly improve the indexing and SEO ranking of your Prestashop e-commerce.
Excessive scanning can also overload server resources, causing slowdowns and critical performance issues, to the point of causing true Denial of Service (DoS) situations, i.e., temporary interruption of service. In an e-commerce environment, where fast page loading significantly affects both user experience and search engine rankings, these problems can translate into a direct drop in sales and a worsening of user satisfaction.
Precisely to limit the negative effects resulting from excessive scanning, we have developed the PrestaShop Security & Bot Shield with Project Honeypot module, which allows us to identify and block requests from malicious bots engaged in web scraping activities and from crawlers used for artificial intelligence training.
The crawling budget represents the maximum number of pages that a search engine, such as Google, decides to crawl a website in a given period of time. Each site has its own crawling budget, which is set based on various factors such as authority, content quality, and technical performance. It is important to manage it properly to prevent crawlers from wasting resources on irrelevant or duplicate pages, instead ensuring that they focus on content that is strategic for ranking and online visibility.
![[Modulo PrestaShop] PrestaShop Security & Bot Shield also...](https://cdn.tecnoacquisti.com/611-home_default/-modulo-prestashop-prestashop-security-bot-shield-also-with-abuseipdb.jpg)
To diagnose any problems related to crawling budgets, it is essential to take advantage of the information provided by Google Search Console. Within the "Crawling" section you can view the "Crawling Statistics" report, which shows how often Googlebot visits your site and the number of daily queries made. Pay particular attention to sudden spikes or drops in the number of pages crawled, which could indicate technical problems or wasted crawling budget. Also, in the "Coverage" section you can spot any errors, duplicate URLs or pages excluded from crawling, all critical issues that negatively affect the way Google explores your site.
Finally, regularly analyze the "Improvements" section to uncover specific issues related to duplicate content, slow pages, or blocked resources that could affect the efficiency of crawling and, consequently, the organic ranking of your e-commerce.
Navigation filters are tools found in e-commerce that allow users to refine their product search by selecting specific attributes such as size, color, price or category. Implementing them, as is done, for example, with Prestashop's Aspect Search (ps_facetedsearch) module, makes the browsing experience more intuitive and improves conversion rates by helping users quickly find what they are looking for.
However, improper use or misconfiguration of filters can cause a large number of different but similar-content URLs to be automatically generated, generating duplicate content problems and wasted crawling budgets. This can significantly penalize SEO, complicate search engine indexing of the site, and reduce the organic visibility of the online store. Therefore, it is essential to carefully manage navigation filters to maximize their benefits without incurring the most common SEO criticalities.
The Aspect Search (ps_facetedsearch) module, also known as Layered Navigation, is a free PrestaShop extension, available on GitHub, that allows users to refine product searches through advanced filters. Despite its usefulness and popularity, this module has historically exhibited several issues, some of which are still not fully resolved today. The main critical issues are:
Given the nature and impact of these issues, we recommend evaluating alternative modules that, while paid, offer superior performance and do not have these critical issues. Among the most reliable and comprehensive alternatives we suggest:
To effectively address the problems associated with unnecessary scans, the first step is to install our PrestaShop Security & Bot Shield module. This tool allows you to automatically block malicious bots and crawlers used to feed AI templates, thus excluding most of the unnecessary requests. In addition, the module already integrates specific modifications to the robots.txt file, designed specifically to mitigate problems arising from the use of the ps_facetedsearch module.
If you prefer to proceed manually, we recommend adding the following directives to the robots.txt file to prevent search engines from indexing dynamically generated pages from the filters:
If you use an alternative module to Aspect Search (ps_facetedsearch), refer to the documentation provided with that module or ask the developer.
Always remember to regenerate the robots.txt file before any changes, via the path:
Finally, in case you are using the ps_facetedsearch module, it is strongly recommended that you constantly update it to the latest available version to benefit from the latest fixes and improvements.
In some cases, despite instructing Googlebot not to scan certain pages, it still continues to access them, often causing server CPU overload and significant performance slowdown, with effects similar to a DoS attack. To avoid this, it is advisable to respond with a 410 code, so as to clearly indicate to the search engine that the URLs should be removed. Below is an example of how to perform this operation:
The Aspects Search module (ps_facetedsearch) uses an internal cache to speed up the display of filtered search results. However, this cache, if not properly managed, tends to grow rapidly and can reach excessive size, causing an overall site slowdown and reduced database performance. Therefore, it is essential to set up frequent cache clearing through scheduled tasks (cron).
In particular, in the case of high-traffic e-commerce, we recommend emptying the cache at least once a day to keep the site fast and responsive. The cron to be set is the one related to the "Empty block cache" operation.
From the Plesk control panel:
Alternatively, via the service easycron.com:
These operations ensure optimal performance of your PrestaShop site and a better user experience, avoiding potential problems resulting from cache overload.
Optimizing the crawling budget of your PrestaShop e-commerce is a crucial aspect of ensuring its optimal performance, search engine visibility, and positive user experience. As we have seen, the Appearance Search module (ps_facetedsearch), while useful and free, has several issues that can negatively affect the indexing and overall performance of your site.
To address these issues, it is essential to regularly monitor resource usage and take preventive action through optimization of the robots.txt file, use of advanced security tools such as the PrestaShop Security & Bot Shield module, and proper cache management through scheduled tasks (cron). However, in more complex cases or in the presence of high traffic, evaluating premium alternatives, such as Amazzing Filter or Advanced Search 5, can be a strategic choice to drastically reduce criticality and technical problems.
Careful and conscious management of these aspects will make a difference not only in technical terms, but especially in the economic performance of your e-commerce, improving conversions and user satisfaction.
Author: Loris Modena
Per Ind Loris Modena owner of Arte e Informatica, began working in the computer industry in 1989 as a systems engineer in charge of maintenance and installation of computer systems. He starts programming for the web in 1997 dealing with CGI programming in PERL and later moving to programming in PHP and JavaScript. During this period he approaches the Open source world and Linux server management.
