How to Extract Information from within a Shadow Root Using Selenium Python

Selenium provides a robust framework for web automation, including the ability to extract information from dynamically loaded web elements such as shadow roots. This guide will demonstrate how to overcome the challenge of extracting product labels and other fields from a specific online store within a shadow root.

The Challenge: Extracting Information from a Shadow Root

When encountering shadow roots, it becomes necessary to explore alternative strategies for element location and extraction. The conventional XPath approach may not suffice. This guide presents a solution that leverages Selenium's execute_script() method to access the shadow root and retrieve the desired information.

Solution: Using ShadowRoot.querySelector()

To extract the product label, utilize the following approach:

driver.get('https://www.tiendasjumbo.co/buscar?q=mani')
item = driver.execute_script("return document.querySelector('impulse-search').shadowRoot.querySelector('div.group-name-brand h1.impulse-title span.formatted-text')")
print(item.text)

Output:

La especial mezcla de nueces, maní, almendras y marañones x 450 g

Conclusion

This solution demonstrates how to effectively extract information from within a shadow root using Selenium Python, providing a reliable technique for automating web interactions even in challenging scenarios.