StatusRU down?

Worklog + Outages

Changelog

DateRegionsAPINetworkChange TypeAPI versionNote
Sep 05, 2022ALLnodeMainNetUPGRADE🆕3.9.2an
3.8.1.an
6k upgrade
Sep 05, 2022ALLidxMainNetUPGRADE🆕2.14.0
2.11.2
6k upgrade
Sep 03, 2022ALLnodeTestNetUPGRADE🆕3.9.2an
3.8.1.an
6k upgrade
Sep 03, 2022ALLidxTestNetUPGRADE🆕2.14.0
2.11.2
6k upgrade
Aug 11, 2022ALLnodeBetaNetUPGRADE🆕3.9.0
3.8.1.an
6k protocol update
Aug 11, 2022ALLidxBetaNetUPGRADE🆕2.14.0
2.11.2
state proofs update
Jul 10, 2022ALLnodeMainNet TestNetUPGRADE🆕3.8.1
3.7.2.an
unscheduled upgrade
Jul 10, 2022ALLidxMainNet TestNetUPGRADE🆕2.11.2
2.11.1
unscheduled upgrade
Jun 20, 2022ALLnodeMainNetUPGRADE🆕3.7.2.an
3.6.2.an
+ early return patch for /pending endpoint
Jun 17, 2022EU/USnodeTestNetFIXno changefix for /v2/transactions/pending/{txid}
randomly returning 404
Jun 15, 2022ALLidxTestNet
MainNet
FIXno changefix for /v2/applications/.../logs
returning data only from the last 20000 blocks
Jun 15, 2022ALLnodeTestNetUPGRADE🆕3.7.2.an
3.6.2.an
+ early return patch for /pending endpoint
Jun 14, 2022ALLnodeBetaNetUPGRADE🆕3.7.2.an
3.6.2.an
+ early return patch for /pending endpoint
May 16, 2022ALLnodeMainNetUPGRADE🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 13, 2022ALLidxTestNet
MainNet
FIXno changefix for 504 timeouts on queries like
/v2/transactions?asset-id=287867876&tx-type=acfg
May 12, 2022ALLnodeTestNetUPGRADE🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 11, 2022ALLnodeBetaNetUPGRADE🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 10, 2022ALLidxMainNetFIXno changefix for 504 timeouts on queries like
mainnet-idx/v2/transactions?application-id=389002307
May 10, 2022ALLidxTestNetFIXno changefix for 504 timeouts on queries like
testnet-idx/v2/transactions?limit=10&tx-type=axfer
May 9, 2022EU / SG / INidxTestNetFIXno changefix truncated indexer returning 500 in some regions
testnet-idx/v2/transactions?
tx-type=acfg&address=...&min-round=21476440
May 5, 2022ALLidxMainNet
TestNet
UPGRADE🆕2.11.1
2.11.0.an
no patches (this time)
+ all parameters enabled
May 4, 2022ALLidxMainNet
TestNet
UPGRADE🆕2.11.0.an
2.10.0.an
+ patch for Exclude parameter overridden…
+ all parameters enabled
  • .an = AlgoNode custom (but published) patches

Outages

📈Realtime Dashboard

Start (UTC)DurationRegionsOutage TypeServicesNote
May 4, 2022
13 DAYS(!)EUstale idxIDX33% of EU requests for Free Indexer API returned stale data
stuck at block 20764300. Our monitoring setup seems to be useless.
May 10, 2022
13:30:45
2 minutesALLloss of datadashboardloss of logs during the outage
will implement log buffering next monitoring epic
May 9, 2022
20:00:04
2 minutesEUcapacitydashboardreduced load-balancer redundancy from 4 to 2 due to hardware fault
no requests lost
May 6, 2022
11:58:46
11 minutesEUcapacityAPI IDXreduced capacity due to hardware fault
no requests lost

Roadmap

DateCategoryStatusNote
Q3'2022OpenSourcealfaAlgoNode Virtual Node. An easy to deploy “proxy” for one or multiple local/on-prem catchup nodes.
Proxy behaves like a full node and forwards queries for archival data to a set of full nodes (eg API provider).
Special handling of local indexer wil full speed historical block feed. Indexer no longer needs for full node.
Custom virtual indexer endpoint that provides streaming blocks in indexer format without the need for indexer
Q4'2022$erviceR&DKafka cluster with JSON(indexer) and MSGpack(node) streaming blocks