StatusRU down?

Worklog + Outages

Changelog

DateRegionsAPINetworkChange TypeAPI versionNote
May 16, 2022
deployed
ALLNodeMainNetAPI upgrade🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 13, 2022ALLIDXTestNet
MainNet
Infra tuningno changefix for 504 timeouts on queries like
/v2/transactions?asset-id=287867876&tx-type=acfg
May 12, 2022ALLNodeTestNetAPI upgrade🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 11, 2022ALLNodeBetaNetAPI upgrade🆕3.6.2.an
3.5.1.an
+ early return patch for /pending endpoint
May 10, 2022ALLIDXMainNetInfra tuningno changefix for 504 timeouts on queries like
mainnet-idx/v2/transactions?application-id=389002307
May 10, 2022ALLIDXTestNetInfra tuningno changefix for 504 timeouts on queries like
testnet-idx/v2/transactions?limit=10&tx-type=axfer
May 9, 2022EU / SG / INIDXTestNetLB cfg fixno changefix truncated indexer returning 500 in some regions
testnet-idx/v2/transactions?
tx-type=acfg&address=...&min-round=21476440
May 5, 2022ALLIDXMainNet
TestNet
API upgrade🆕2.11.1
2.11.0.an
no patches (this time)
+ all parameters enabled
May 4, 2022ALLIDXMainNet
TestNet
API upgrade🆕2.11.0.an
2.10.0.an
+ patch for Exclude parameter overridden…
+ all parameters enabled
  • .an = AlgoNode custom (but published) patches

Outages

📈Realtime Dashboard

Start (UTC)DurationRegionsOutage TypeServicesNote
May 4, 2022
13 DAYS(!)EUstale idxIDX33% of EU requests for Free Indexer API returned stale data
stuck at block 20764300. Our monitoring setup seems to be useless.
May 10, 2022
13:30:45
2 minutesALLloss of datadashboardloss of logs during the outage
will implement log buffering next monitoring epic
May 9, 2022
20:00:04
2 minutesEUcapacitydashboardreduced load-balancer redundancy from 4 to 2 due to hardware fault
no requests lost
May 6, 2022
11:58:46
11 minutesEUcapacityAPI IDXreduced capacity due to hardware fault
no requests lost
May 2022mitigatedINlatencyAPIBangalore is constantly loosing connectivity to (redirected) Cloudflare.
FrontEnd endpoints are redirected to Singapore.
Back-end endpoints are fine.

Roadmap

DateCategoryStatusNote
Jul 2022OpenSourcealfaAlgoNode Virtual Node. An easy to deploy “proxy” for one or multiple local/on-prem catchup nodes.
Proxy behaves like a full node and forwards queries for archival data to a set of full nodes (eg API provider).
Special handling of local indexer wil full speed historical block feed. Indexer no longer needs for full node.
Custom virtual indexer endpoint that provides streaming blocks in indexer format without the need for indexer
Jul 2022Free APItestingAU Region
Aug 2022Free APIplanningSouth Africa Region
Aug 2022Free APIplanningBrasil Region
Aug 2022$erviceR&DKafka cluster with JSON(indexer) and MSGpack(node) streaming blocks
Aug 2022Free API $erviceplanningSingapore upgrade to full Region / commercial