Fetch restore request based on last fetched request

[chkp-arvidb]

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

• In Progress
• Ready
• In Hold - (Reason for hold)

Related Issues

relates: https://jira-dc.paloaltonetworks.com/browse/CIAC-17782

Description

Fixes an issue in the Check Point Harmony Email and Collaboration (HEC) integration where restore request incidents could be missed during fetch.

Previously, fetch_restore_requests advanced a single shared cursor to the newest fetched record (or to "now" on empty fetches), so restore requests that were not yet available in the search
index — or that belonged to a less busy SaaS app — could be skipped permanently.

Changes:

• The restore request cursor (last_rr_fetch) is now tracked per SaaS app, so a busy app no longer drags the cursor forward past unfetched records of another app. Existing single-cursor
  state is migrated on upgrade.
• Each app's cursor only advances to the restoreRequestTime of the newest incident actually emitted for that app; apps with no results keep their previous cursor (bounded by a 1-day max
  look-back).
• Added scroll-based pagination to the restore requests query so more than one page of results can be collected, and results from all apps are merged in ascending order before applying
  max_fetch, keeping the oldest records on truncation.
• Added debug logging to fetch_incidents and fetch_restore_requests for troubleshooting fetch windows, cursors, and truncation.

Must have

• Tests
• Documentation

[content-bot]

Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @kamalq97 will know the proposed changes are ready to be reviewed.
For your convenience, here is a link to the contributions SLAs document.

[content-bot]

Hi @chkp-arvidb, thanks for contributing to the XSOAR marketplace. To receive credit for your generous contribution please follow this link.

[content-bot]

🤖 AI-Powered Code Review Available

Hi @kamalq97, you can leverage AI-powered code review to assist with this PR!

Available Commands:

• @marketplace-ai-reviewer start review - Initiate a full AI code review
• @marketplace-ai-reviewer re-review - Incremental review for new commits