Client Side Integration - Client Spider

This add-on adds a Client Spider which is designed to explore modern web apps more effectively.

The Client Spider works in a similar way to the AJAX Spider but it has access to the DOM via the ZAP Browser Extension which means that it can find content which the AJAX Spider cannot find.

The Client Spider supports all of the authentication options supported by ZAP, and will run any enabled Selenium scripts in the browsers that it launches.

While it is still at an early stage we believe it is a more effective approach than the AJAX Spider.

We will be focussing on improving the Client Spider and the current plan is for it to supersede the AJAX Spider as the recommended way of crawling modern web apps.

We would appreciate feedback via the ZAP User Group to let us know how effectively it works for you, especially in comparison with the AJAX Spider.

The spider can be invoked via:

Context specific “Attack” menu
“Tools / Client Spider” menu item
Automation Framework spiderClient job

Client Spider dialog

The dialog for starting the Client Spider provides 2 sub tabs:

Scope

This tab allows you to define what the spider will attempt to explore.

The Scope Check can be either:

Strict - enforces that all requests need to be in scope to be accessed.
Flexible - allows all requests to be accessed. This scope check has the side effect of allowing out of scope domains to be accessed, but not crawled.

Options

This tab allows you to define the options that control how the spider works.

Number of Browser Windows to Open - The number of browser windows the spider uses in parallel. More windows increases speed but uses more memory.
Maximum Crawl Depth - The maximum depth the spider will crawl to. 0 means unlimited.
Maximum Children - The maximum number of child nodes to add under any single node in the tree. 0 means unlimited.
Initial Page Load Time - The number of seconds to wait after the initial URL is loaded. Default: 5.
Page Load Time - The number of seconds to wait after each subsequent URL is loaded. Default: 1.
Action Wait Time - The number of seconds to wait after performing an action on the browser (e.g. opening a URL, clicking a button, submitting a form). Default: 0.
Shutdown Time - The number of seconds to wait for new events after the last action before the spider shuts down. Default: 5.
Maximum Duration - The maximum number of minutes the spider is allowed to run. 0 means unlimited.
Logout Avoidance - When enabled, the spider will avoid clicking elements that are likely to log the user out.

Client Spider tab

The Client Spider tab allows you start and monitor the Client Spider. It provides 3 sub tabs:

Added Nodes

These are the nodes that have been added to the Client Map.

Tasks

These are the tasks that spider uses to crawl the application. The tasks are updated when they are added to the task list, when they start running, and when they complete. This allows you to understand what the Client Spider is actually doing much more clearly.

Messages

These are the HTTP(S) messages sent from the browsers that the client uses.