AI-powered crawl trap prevention & Document categorisation

Two new AI capabilities to make customer implementations lighter than air

We continue to make Airgentic implementations lighter than air for our customers. In this release we have added two AI-automated capabilities that reduce manual effort, speed time to value and improve the reliability of production crawls and search.

AI-powered crawl trap prevention

Crawlers can waste large amounts of time in crawl traps created by dynamically generated pages that add no value to search or automation. Historically, diagnosing and resolving these traps has required an expert consultant to adjust configuration and get the crawl back on track.

Airgentic now automates that work. Our crawler self-monitors for signs of a trap, determines an appropriate response and updates its configuration so it returns quickly to useful content. The result is fewer manual interventions, reduced wasted crawl time and more predictable, reliable implementations.

AI-based document categorisation for better search

Great search often depends on consistent tagging and categorisation, which can mean manually classifying hundreds or thousands of pages so results are grouped and ranked in a way that makes sense to customers.

Airgentic now categorises pages automatically before indexing. Pages are assigned types such as help content, product information, policies, forms and news, enabling search results to be organised and tuned without manual tagging. This reduces manual tuning, accelerates implementations and helps deliver AI customer service without heavy professional services overhead.