Skip to content

Horizon: improve logging when config is broken #97

Description

@jacekn

What problem does your feature solve?

It would make troubleshooting easier

What would you like to see?

I had misconfigured horizon where it was unable to reach history archive defined using the HISTORY_ARCHIVE_URLS variable.
Horizon started but it didn't bring up captivecore.
In the logs I had lots of messages similar to:

time="2024-07-17T11:55:02.620Z" level=error msg="failed to load the stellar-core info" err="http request errored: Get \"http://127.0.0.1:11626/info\": dial tcp 127.0.0.1:11626: connect: connection refused" pid=221079 stack="[main.go:43 client.go:139 app.go:239 app.go:459 asm_amd64.s:1695]"
time="2024-07-17T11:55:03.620Z" level=warning msg="could not load stellar-core info: http request errored: Get \"http://127.0.0.1:11626/info\": dial tcp 127.0.0.1:11626: connect: connection refused" pid=221079

which made it harder to spot root cause which was only logged once:

time="2024-07-17T11:54:59.638Z" level=error msg="Error in ingestion state machine" current_state="resume(latestSuccessfullyProcessedLedger=2658758)" error="error preparing range: error starting prepare range: opening subprocess: error calculating ledger and hash for stellar-core run: error getting latest checkpoint sequence: error getting root HAS: Get \"http://<redacted>/.well-known/stellar-history.json\": dial tcp <redacted>:80: i/o timeout" next_state=start pid=221079 service=ingest

I think it would be worth suppressing the captivecore connection refused messages if horizon is stuck talking to the history archives. If the history archive message was the last one logged I think it would be easier to spot root cause in the logs.

What alternatives are there?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions