Fix NFT page showing only Ethereum: IPFS gateway fallback, show-more, Solana wallet changes

.env.example

@@ -249,6 +249,10 @@ NFT_IMAGE_CACHE_ENABLED=false
 NFT_IMAGE_MAX_SIZE_MB=5
 NFT_IMAGE_THUMBNAIL_SIZE=150
 
+# IPFS gateways the NFT page uses for images, tried in order (comma-separated,
+# https:// only, each ending in /ipfs/). Leave unset for the built-in list.
+# NFT_DISPLAY_IPFS_GATEWAYS=https://ipfs.blockfrost.dev/ipfs/,https://gateway.pinata.cloud/ipfs/,https://ipfs.io/ipfs/,https://dweb.link/ipfs/
+
 #-------------------------------------------------------------------------------
 # OPTIONAL: NFT Background Scheduler (v0.9.0+)
 #-------------------------------------------------------------------------------

CHANGELOG.md

@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **BTC Karma Bitcoin staking card (DeFi page)**: tracked Bitcoin wallets are now scanned for [BTC Karma](https://staking.btckarma.io) stakes, shown as a "Bitcoin · BTC Karma" card under Staked Positions with the staked BTC, its USD value and the Cardano wallet that receives the KARMA rewards. Data comes straight from the Bitcoin chain through the public Esplora API (mempool.space, blockstream.info fallback) — no BTC Karma login or key. A stake is a transaction spent from your address that pays a P2WSH vault and carries an `OP_RETURN` of `CB|` + blake2b-512 of the bound Cardano address; an unspent vault is an active stake, so unstaking removes the card on its own. To use it, add the Bitcoin address you staked from as an ordinary Bitcoin wallet (single address, not an xpub). Unlock time and KARMA reward balances are not public on-chain and are not shown. New endpoint `GET /defi/btc-karma/{address}` (1 h cache, stale-cache fallback, shows "Data unavailable" rather than erroring). 24 new unit tests in `tests/unit/test_btc_karma.py` (recorded mainnet responses, no live calls).
 
 ### Fixed
+- **NFT page showed only Ethereum after the Cardano image fix**: #12 pointed every Cardano NFT image at the public `ipfs.io` gateway, which now answers hot-linked images with a Cloudflare challenge (HTTP 403/429). All Cardano (and the Algorand) images failed; Gallery Mode hides tiles whose image fails, so only Ethereum was left, and the grid showed placeholders. IPFS images are now offered on several gateways (`image_url` plus ordered `image_fallbacks`, configurable with `NFT_DISPLAY_IPFS_GATEWAYS`, default `ipfs.blockfrost.dev` → Pinata → `ipfs.io` → `dweb.link`) and the page tries the next gateway before giving up. The page only ever uses `https://` / `data:image/` image URLs (closes the raw `http://` fallback noted in the #12 review). The All Chains grid no longer stops at 100 cards (a "Show more" button reveals the rest, so Algorand/Solana are reachable), collection names no longer render as `[object Object]`, and the collection count is per chain. Solana NFTs are refetched when a Solana wallet is added or removed (a hash of the wallet set is stored with the cache) instead of staying hidden for up to 24 h / 30 days. 36 new unit tests in `tests/unit/test_nft_display_images.py`.
 - **Solana wallet balances failed, and Settings showed Helius as failing, even though Helius NFTs worked (ABCT-HELIUS-BALANCES-FIX-2026-09-28)**: Helius retired the v0 REST `/addresses/{address}/balances` endpoint (now HTTP 404 "Method not found"), which `services/solana.py` used for every Solana wallet's SOL + SPL token balance, and which the Settings page's Helius health/test check also called — so Solana balances failed on every wallet (`wallet_balance_bgsync`: "Failed to fetch balance from all sources (Helius + public RPC)", 21× in the logs) and a newly added Solana wallet showed 0 SOL and 0 tokens even with real holdings, while Helius NFT (DAS) fetches kept working fine. `get_address_info()` now uses the Helius DAS `searchAssets` JSON-RPC method (`tokenType: "fungible"`, `displayOptions.showNativeBalance`) — the current, GA, free-tier-compatible way to get native SOL + SPL token balances in one call ([docs](https://www.helius.dev/docs/api-reference/das/searchassets)) — instead of the retired endpoint; same output shape (`balance_sol`, `balance_lamports`, `tokens[]` with mint/symbol/name/balance/amount_raw/decimals), same DB-first-then-env key lookup, same public-RPC fallback on a network-level failure. The Settings "helius" health/test check now POSTs a plain, address-independent Solana `getHealth` JSON-RPC call instead of hitting the retired endpoint. No other chain's balance path used this endpoint. 11 new unit tests in `tests/unit/test_helius_balances_fix.py` (mocked HTTP/DB, fake addresses, no live calls or real keys), including SOL-only, SOL+SPL (real HNT mint/decimals), zero-balance, provider-error and health-check cases, plus an end-to-end test through the real wallet-refresh save path.
 - **System-wide cache entries piled up and were read back stale (ABCT-NFT-DIAG-2026-09-27)**: the `cache` table's `UNIQUE(user_id, key)` never matches system-wide entries, because SQLite treats NULLs as distinct. `set_cache(..., user_id=None)` therefore inserted a new row on every write instead of updating, and `get_cache` read the oldest row. A production snapshot had 1,057 system-wide rows for 127 keys. The persistent Ethereum/Polygon/Base/Solana NFT caches (and every other system-wide cache) were read back as long-expired entries, so each restart re-fetched from the providers. System-wide writes now update in place (inserting only when absent, and bringing every legacy duplicate up to date), and reads take the newest row. No schema change. 3 new unit tests in `tests/unit/test_system_cache_upsert.py`.
 - **Iagon staking scan could permanently drop a transaction on a transient fetch failure (ABCT-IAGON-SCANSTATE-FIX-2026-09-28)**: both Iagon staking scan implementations (`services/defi_protocols/cardano/iagon.py`, `services/defi.py`) fetch each transaction's UTxOs in parallel and skip any transaction whose fetch failed (HTTP error or exception) for that call, but still saved the incremental scan cursor (`iagon_scan_state_{address}`) past it whenever a later transaction succeeded — so the skipped transaction's IAG flow was never retried and the staked IAG total (shown in DeFi and portfolio) could drift and never self-heal. The scan now only saves/advances its state when every UTxO fetch in that run succeeded; a run with any failure leaves the previous state untouched so the next scan retries the same range. This is a latent bug on `main` today (the system-wide cache is effectively always a full rescan) but becomes load-bearing once the system-wide cache fix lands. `SCAN_STATE_VERSION` bumped 5→6 in both implementations (follow-up from review) so the first scan after deploy discards any pre-existing v5 row automatically — including one that may itself already be holed by this bug — instead of needing a manual "Clear staking cache" after deploy. 8 new unit tests in `tests/unit/test_iagon_scan_state_failure.py` (mocked HTTP and cache; no live calls or real keys), covering both implementations, including a stale-v5-row rescan test.

abct-docker/.env.example

@@ -64,6 +64,10 @@ CMC_API_KEY=
 # Enable local caching of NFT images
 NFT_IMAGE_CACHE_ENABLED=false
 
+# IPFS gateways the NFT page uses for images, tried in order (comma-separated,
+# https:// only, each ending in /ipfs/). Leave unset for the built-in list.
+# NFT_DISPLAY_IPFS_GATEWAYS=https://ipfs.blockfrost.dev/ipfs/,https://gateway.pinata.cloud/ipfs/,https://ipfs.io/ipfs/,https://dweb.link/ipfs/
+
 # ===========================================
 # OPTIONAL SERVICES
 # ===========================================

backend/config.py

@@ -301,6 +301,22 @@
     "https://gateway.pinata.cloud/ipfs/",
 ]
 
+# IPFS gateways used for NFT images shown in the browser (ABCT-NFT-DIAG2-2026-09-28).
+# The page tries them in order and falls back to the next one when an image
+# fails to load. ipfs.io answers hot-linked images with a Cloudflare challenge
+# (HTTP 403/429), so it is no longer first. Override with a comma-separated
+# list of https:// gateway prefixes ending in "/ipfs/".
+_DEFAULT_NFT_DISPLAY_IPFS_GATEWAYS = (
+    "https://ipfs.blockfrost.dev/ipfs/,"
+    "https://gateway.pinata.cloud/ipfs/,"
+    "https://ipfs.io/ipfs/,"
+    "https://dweb.link/ipfs/"
+)
+NFT_DISPLAY_IPFS_GATEWAYS = [
+    g.strip() for g in os.getenv("NFT_DISPLAY_IPFS_GATEWAYS", _DEFAULT_NFT_DISPLAY_IPFS_GATEWAYS).split(",")
+    if g.strip().lower().startswith("https://")
+]
+
 # NFT Background Scheduler Configuration
 NFT_SCHEDULER_ENABLED = os.getenv("NFT_SCHEDULER_ENABLED", "false").lower() == "true"
 NFT_UPDATE_INTERVAL_MINUTES = int(os.getenv("NFT_UPDATE_INTERVAL_MINUTES", "15"))

backend/routers/nfts.py

@@ -13,6 +13,7 @@
 
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from services.nft import nft_service
+from services.nft_display_images import attach_display_images
 from services.nft_image_service import (
     nft_image_service,
     MAX_BATCH_ITEMS,
@@ -845,6 +846,11 @@ async def get_algorand_nfts(user_id: int = Depends(verify_session), force_refres
         except Exception as e:
             logger.error(f"Error fetching Algorand NFTs for {wallet['address'][:12]}...: {e}")
 
+    # Offer IPFS images on several gateways, like Cardano (ABCT-NFT-DIAG2-2026-09-28)
+    for nft in all_nfts:
+        if isinstance(nft, dict):
+            attach_display_images(nft, nft.get('image_url'))
+
     return {
         'configured': True,
         'nfts': all_nfts,

backend/services/nft.py

@@ -34,6 +34,7 @@
 )
 from services.http_client import get_client, blockfrost_fetch
 from services.api_key_manager import APIKeyManager
+from services.nft_display_images import attach_display_images
 
 # DB-stored key (Settings page) first, env var fallback — matches every
 # other Blockfrost caller in the app. See ABCT-BF-KEY-UNIFY-2026-09-27.
@@ -842,8 +843,11 @@ def attach_image_urls(self, nfts: List[dict], user_id: Optional[int] = None) ->
         for nft in nfts:
             if not isinstance(nft, dict):
                 continue
-            url = self._safe_display_url(nft.get('image'))
-            nft['image_url'] = url
+            # IPFS images are offered on several gateways (image_url + ordered
+            # image_fallbacks): the public ipfs.io gateway alone now blocks
+            # hot-linked images (ABCT-NFT-DIAG2-2026-09-28).
+            attach_display_images(nft, nft.get('image'))
+            url = nft['image_url']
             if not url and not nft.get('image_checked') and nft.get('asset_id'):
                 missing.append(nft['asset_id'])
         if missing and user_id is not None:

backend/services/nft_display_images.py

@@ -0,0 +1,104 @@
+"""
+Browser-facing NFT image URLs (ABCT-NFT-DIAG2-2026-09-28).
+
+NFT metadata usually points at IPFS, and ABCT used to hand the browser a
+single ``https://ipfs.io/ipfs/<cid>`` URL. The public ipfs.io gateway now
+answers hot-linked images with a Cloudflare challenge (HTTP 403/429), so every
+Cardano (and the Algorand) NFT image failed to load and the NFT page's Gallery
+Mode, which hides tiles whose image fails, showed only Ethereum NFTs.
+
+``display_image_candidates()`` turns one metadata image URL into an ordered
+list of URLs the browser may try: IPFS content is offered on every configured
+gateway (``NFT_DISPLAY_IPFS_GATEWAYS``), anything else is passed through only
+if it is ``https://`` or an inline ``data:image/`` URI. On-chain metadata is
+attacker-controlled, so ``http://`` and other schemes are never handed out.
+"""
+
+import re
+from typing import List, Optional
+from urllib.parse import quote, unquote, urlsplit
+
+try:
+    from config import NFT_DISPLAY_IPFS_GATEWAYS
+except Exception:  # pragma: no cover - config import problems must not break NFT lists
+    NFT_DISPLAY_IPFS_GATEWAYS = ["https://ipfs.io/ipfs/"]
+
+# CIDv0 (Qm + 44 base58 chars) or CIDv1 in base32 (bafy..., bafk..., bafb...)
+_CID_RE = re.compile(r"^(Qm[1-9A-HJ-NP-Za-km-z]{44}|baf[a-z2-7]{20,})$")
+# Characters left as-is in the path after the CID (file names inside a
+# directory CID); everything else is percent-encoded.
+_PATH_SAFE_CHARS = "/!$&'()*+,;=:@-._~"
+
+
+def ipfs_path(url) -> Optional[str]:
+    """Return ``<cid>[/path]`` if ``url`` refers to IPFS content, else None.
+
+    Recognises ``ipfs://<cid>``, ``ipfs://ipfs/<cid>``, a bare CID, and any
+    ``http(s)://<gateway>/ipfs/<cid>`` gateway URL.
+    """
+    if not isinstance(url, str):
+        return None
+    u = url.strip()
+    if not u:
+        return None
+    lowered = u.lower()
+    if lowered.startswith("ipfs://"):
+        rest = u[7:]
+        if rest.lower().startswith("ipfs/"):
+            rest = rest[5:]
+    elif lowered.startswith(("https://", "http://")):
+        try:
+            path = urlsplit(u).path
+        except ValueError:
+            return None
+        if not path.startswith("/ipfs/"):
+            return None
+        rest = path[len("/ipfs/"):]
+    else:
+        rest = u
+    rest = rest.lstrip("/")
+    cid, _, sub = rest.partition("/")
+    if not _CID_RE.match(cid):
+        return None
+    if sub:
+        sub = sub.split("?", 1)[0].split("#", 1)[0]
+        decoded = unquote(sub)
+        if any(seg == ".." for seg in decoded.split("/")) or any(ord(ch) < 32 for ch in decoded):
+            return None
+        sub = quote(decoded, safe=_PATH_SAFE_CHARS)
+    return cid + ("/" + sub if sub else "")
+
+
+def _safe_passthrough(url) -> Optional[str]:
+    if not isinstance(url, str) or not url.strip():
+        return None
+    u = url.strip()
+    lowered = u.lower()
+    if lowered.startswith("https://") or lowered.startswith("data:image/"):
+        return u
+    return None
+
+
+def display_image_candidates(url, gateways: Optional[List[str]] = None) -> List[str]:
+    """Ordered, de-duplicated list of browser-safe URLs for one NFT image."""
+    gateways = NFT_DISPLAY_IPFS_GATEWAYS if gateways is None else gateways
+    path = ipfs_path(url)
+    candidates: List[str] = []
+    if path:
+        for g in gateways:
+            if isinstance(g, str) and g.lower().startswith("https://"):
+                candidates.append(g.rstrip("/") + "/" + path)
+    else:
+        safe = _safe_passthrough(url)
+        if safe:
+            candidates.append(safe)
+    seen = set()
+    return [c for c in candidates if not (c in seen or seen.add(c))]
+
+
+def attach_display_images(nft: dict, source_url) -> dict:
+    """Set ``image_url`` (first candidate or None) and ``image_fallbacks`` on ``nft``."""
+    candidates = display_image_candidates(source_url)
+    nft["image_url"] = candidates[0] if candidates else None
+    nft["image_fallbacks"] = candidates[1:]
+    return nft

backend/services/solana_nft.py

@@ -10,6 +10,7 @@
 Uses persistent database caching to reduce API calls and survive restarts.
 """
 
+import hashlib
 import httpx
 import logging
 from typing import Dict, List, Optional
@@ -30,6 +31,17 @@
 SOL_NFT_CACHE_TTL = 86400 * 30  # 30 days - persistent until manual refresh
 
 
+def wallet_fingerprint(addresses) -> str:
+    """Stable, non-reversible fingerprint of a set of wallet addresses.
+
+    Stored next to the cached NFTs so a wallet added or removed after the
+    last fetch is noticed (ABCT-NFT-DIAG2-2026-09-28). Only a hash is kept,
+    never the addresses themselves.
+    """
+    joined = "\n".join(sorted({a for a in addresses if a}))
+    return hashlib.sha256(joined.encode("utf-8")).hexdigest()
+
+
 class SolanaNFTService(APIKeyManager):
     """Service for fetching Solana NFTs from Helius DAS API."""
 
@@ -41,6 +53,8 @@ def __init__(self):
         self._cache_ttl = timedelta(hours=24)  # In-memory cache validity
         self.last_refresh: Optional[datetime] = None
         self._db_cache_loaded = False
+        # Fingerprint of the wallet set the cached NFTs were fetched for
+        self._wallet_fingerprint: Optional[str] = None
 
     async def is_configured(self) -> bool:
         """Check if the API key is configured."""
@@ -256,18 +270,42 @@ async def get_all_solana_nfts(self, user_id: int = None, force_refresh: bool = F
         Returns:
             List of all NFTs across all Solana wallets
         """
+        # A cache built for a different set of Solana wallets (one was added or
+        # removed since) is stale no matter how young it is: without this, a
+        # newly added wallet's NFTs stayed hidden for up to 24 h (in-memory TTL)
+        # or 30 days (persistent cache). ABCT-NFT-DIAG2-2026-09-28.
+        current_fp = None
+        if not force_refresh:
+            try:
+                wallets_now = await get_all_wallets(user_id=user_id)
+                current_fp = wallet_fingerprint(
+                    w['address'] for w in wallets_now if w.get('blockchain') == 'solana'
+                )
+            except Exception as e:
+                logger.debug(f"Could not fingerprint Solana wallets: {e}")
+
         # Try to load from persistent database cache first
         if not force_refresh and not self._db_cache_loaded:
             cached_data = await get_cache(SOL_NFT_CACHE_KEY)
             if cached_data:
-                logger.info(f"Loaded {len(cached_data.get('nfts', []))} Solana NFTs from persistent cache")
                 self._nft_cache = {nft['asset_id']: nft for nft in cached_data.get('nfts', [])}
                 self._collection_cache = cached_data.get('collections', {})
                 self.last_refresh = datetime.fromisoformat(cached_data['last_refresh']) if cached_data.get('last_refresh') else None
+                self._wallet_fingerprint = cached_data.get('wallet_fingerprint')
                 self._db_cache_loaded = True
-                return list(self._nft_cache.values())
+                if current_fp is None or self._wallet_fingerprint == current_fp:
+                    logger.info(f"Loaded {len(cached_data.get('nfts', []))} Solana NFTs from persistent cache")
+                    return list(self._nft_cache.values())
+                logger.info("Solana wallets changed since the persistent NFT cache was saved - refetching")
+                force_refresh = True
             self._db_cache_loaded = True
 
+        if (not force_refresh and current_fp is not None
+                and self._wallet_fingerprint is not None
+                and self._wallet_fingerprint != current_fp):
+            logger.info("Solana wallets changed since the last NFT fetch - refetching")
+            force_refresh = True
+
         # Check in-memory cache validity
         if not force_refresh and self._is_cache_valid():
             return list(self._nft_cache.values())
@@ -286,6 +324,7 @@ async def get_all_solana_nfts(self, user_id: int = None, force_refresh: bool = F
 
         all_nfts = []
         self._nft_cache.clear()
+        fetched_fp = wallet_fingerprint(w['address'] for w in sol_wallets)
 
         for wallet in sol_wallets:
             address = wallet['address']
@@ -327,6 +366,7 @@ async def get_all_solana_nfts(self, user_id: int = None, force_refresh: bool = F
             logger.info(f"Fetched {total_fetched} NFTs for Solana wallet {address[:8]}...")
 
         self.last_refresh = datetime.now()
+        self._wallet_fingerprint = fetched_fp
 
         # Save to persistent database cache
         await self._save_to_db_cache()
@@ -339,7 +379,8 @@ async def _save_to_db_cache(self) -> None:
             cache_data = {
                 'nfts': list(self._nft_cache.values()),
                 'collections': self._collection_cache,
-                'last_refresh': self.last_refresh.isoformat() if self.last_refresh else None
+                'last_refresh': self.last_refresh.isoformat() if self.last_refresh else None,
+                'wallet_fingerprint': self._wallet_fingerprint,
             }
             await set_cache(SOL_NFT_CACHE_KEY, cache_data, SOL_NFT_CACHE_TTL)
             logger.info(f"Saved {len(self._nft_cache)} Solana NFTs to persistent cache")
@@ -404,6 +445,7 @@ def clear_cache(self):
         self._collection_cache.clear()
         self.last_refresh = None
         self._db_cache_loaded = False
+        self._wallet_fingerprint = None
 
     async def get_status(self) -> dict:
         """Get service status and configuration."""