⚽ Live Match: docs/learn/fundamentals/data format/xdr

Stellar stores and communicates ledger data, transactions, results, history, and messages in a binary format called External Data Representation (XDR). XDR is defined in RFC4506. XDR is optimized for network performance but is not human-readable. The Stellar SDKs convert XDRs into friendlier formats.

.X files

Data structures in XDR are specified in .x files. These files only contain data structure definitions, no operations or executable code. The .x files for the XDR structures used on the Stellar Network are available on GitHub.

Stellar XDR is encodable into JSON using the XDR-JSON schema.

More About XDR

XDR is similar to tools like Protocol Buffers or Thrift. XDR provides a few important features:

It is very compact, so it can be transmitted quickly and stored with minimal disk space.
Data encoded in XDR is reliably and predictably stored. Fields are always in the same order, which makes cryptographically signing and verifying XDR messages simple.
XDR definitions include rich descriptions of data types and structures, which is not possible in simpler formats like JSON, TOML, or YAML.

Parsing XDR

Since XDR is a binary format and not as widely known as simpler formats like JSON, the Stellar SDKs all include tools for parsing XDR and will do so automatically when retrieving data. The XDR data is still included (encoded as a base64 string) inside the JSON in case you need direct access to it.

Digging into XDR structures

Since the XDR format is a fundamental underpinning of the Stellar Network, we often find it necessary to dig into these raw binary data structures for clients to interact with things like transactions and authorization entries. It can be hard to understand because of its structure, but this section will aim to enlighten you on the ways to interact with it in a handful of popular languages. The Protocol’s schema is defined in the stellar/stellar-xdr repository, though that contains far more than what we’ll need for our purposes.

XDR’s Common Forms

In the Stellar Protocol, XDR takes a handful of different forms that are important to understand and distinguish:

Basic Primitives

These are the easiest to understand for anyone familiar with a programming language and includes things like integers, strings, and arrays of bytes. These are fairly straightforward to interact with, though things can get more complicated when you have aliases. For example, you have the Uint64 alias in the Stellar Protocol (defined Stellar-types.x), defined for readability instead of XDR’s native “unsigned hyper” value. In the JavaScript SDK (v17 and later), a Uint64 is a native BigInt. The xdr.Uint64 helper is a function, not a class: call it without new, and it checks the range and returns a bigint:

import { xdr } from "@stellar/stellar-sdk";

const u64 = xdr.Uint64(12345678); // 12345678n

Some language variations will let you create it in a number of ways. For example, since 64-bit integers are a little… wonky in JavaScript, the helper also accepts a BigInt or a string of digits. A plain bigint literal works too:

import { xdr } from "@stellar/stellar-sdk";

let u64 = xdr.Uint64("1000000000000000");
u64 = xdr.Uint64(1_000_000_000_000_000n);
u64 = 1_000_000_000_000_000n;

SDK v16 and earlier wrap 64-bit integers in classes, so older code calls new xdr.Uint64(...). In v17, new xdr.Uint64(...) and new xdr.Int64(...) throw a TypeError. Learn more in the v17 XDR migration guide.

For the other XDR types, see the v17 XDR migration guide. It shows the v17 shapes of unions, enums, integers, and bytes, so it should be your primary guide to navigating the XDR if you’re using JavaScript. Python, Go, and Java have no such variations. Python (see stellar-sdk’s uint64.html documentation) supports arbitrarily-sized integers by default. In Go, xdr.Uint64 is a named uint64 type, so a plain conversion works. In Java, Uint64 wraps an XdrUnsignedHyperInteger, which takes a Long or, for values above Long.MAX_VALUE, a BigInteger:

from stellar_sdk import xdr

u64 = xdr.Uint64(1_000_000_000_000_000)

import "github.com/stellar/go-stellar-sdk/xdr"

u64 := xdr.Uint64(1_000_000_000_000_000)

import org.stellar.sdk.xdr.Uint64;
import org.stellar.sdk.xdr.XdrUnsignedHyperInteger;

Uint64 u64 = new Uint64(new XdrUnsignedHyperInteger(1_000_000_000_000_000L));

As a result, many of the basic primitives can be initialized in the “intuitive” way for your respective language.

Unions

This is where things get a little more complicated. A union type is a generic type that switches between different “arms”, each of which is a different internal type. When parsing an XDR union type, it’s important to determine which arm is being used before you try parsing what’s inside. Let’s take an SCAddress, for example. From the type definition, we can see that it’s either an account (i.e. SC_ADDRESS_TYPE_ACCOUNT) or the hash of a contract (SC_ADDRESS_TYPE_CONTRACT). Right above that definition, you can see that these are constants of the SCAddressType enum defined as 0 or 1, respectively. In general, unions follow a particular pattern: switch on the type and match the arm you expect. So when you have one of these unions, you need to figure out which one it is:

import { xdr } from "@stellar/stellar-sdk";

// suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
switch (scAddr.type) {
  case "scAddressTypeAccount":
    console.log("The account is:", scAddr.accountId);
    break;

  case "scAddressTypeContract":
    console.log("The contract is:", scAddr.contractId);
    break;

  default:
    throw new Error(`Unexpected address type: ${scAddr.type}`);
}

from stellar_sdk.xdr import SCAddressType

if scAddr.type == SCAddressType.SC_ADDRESS_TYPE_ACCOUNT:
    print("The account is:", scAddr.account_id)

elif scAddr.type == SCAddressType.SC_ADDRESS_TYPE_CONTRACT:
    print("The contract is:", scAddr.contract_id)

else:
    raise Exception(f"Unexpected address type: {scAddr.type}")

import (
  "fmt"

  "github.com/stellar/go-stellar-sdk/xdr"
)

// suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
switch scAddr.Type {
case xdr.ScAddressTypeScAddressTypeAccount:
  fmt.Println("The account is:", scAddr.MustAccountId().Address())

case xdr.ScAddressTypeScAddressTypeContract:
  contractID := scAddr.MustContractId()
  fmt.Printf("The contract is: %x\n", contractID[:])

default:
  panic(fmt.Sprintf("Unexpected address type: %v", scAddr.Type))
}

import org.stellar.sdk.KeyPair;
import org.stellar.sdk.StrKey;

// suppose `scAddr` is an instance of `org.stellar.sdk.xdr.SCAddress` we're parsing
switch (scAddr.getDiscriminant()) {
  case SC_ADDRESS_TYPE_ACCOUNT:
    KeyPair account = KeyPair.fromXdrPublicKey(scAddr.getAccountId().getAccountID());
    System.out.println("The account is: " + account.getAccountId());
    break;

  case SC_ADDRESS_TYPE_CONTRACT:
    byte[] contractId = scAddr.getContractId().getContractID().getHash();
    System.out.println("The contract is: " + StrKey.encodeContract(contractId));
    break;

  default:
    throw new IllegalArgumentException("Unexpected address type: " + scAddr.getDiscriminant());
}

In the JavaScript SDK (v17 and later), .type is the name of the arm as a string, and the arm’s payload is a plain property named after it (scAddr.accountId). TypeScript narrows the union inside each case, so it knows which property exists. Every arm also exposes its payload as .value. In Go, Type is an enum constant and each arm is a pointer field (scAddr.AccountId). The generated MustAccountId() returns the arm and panics if the union holds another one, while GetAccountId() returns it with an ok flag. In Java, getDiscriminant() returns an enum constant and each arm has its own getter (scAddr.getAccountId()), which returns null when the union holds another arm. You can see the references for these in the documentation (Python, JavaScript, Go, and Java, respectively). If you only expect one arm, the JavaScript SDK has two helpers that narrow the union for you:

import { xdr } from "@stellar/stellar-sdk";

// suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
if (xdr.isUnionVariant(scAddr, "scAddressTypeContract")) {
  console.log("The contract is:", scAddr.contractId);
}

// throws a TypeError if `scAddr` is not an account address
const { accountId } = xdr.expectUnionVariant(scAddr, "scAddressTypeAccount");

Underneath, each arm is still identified by an enum value (0 for an account and 1 for a contract here). Matching on those numbers is more error-prone and harder to read than matching on the arm names. Unions appear in many places throughout the XDR and they can be just one arm or a whole bunch of arms. For example, the smart contract value ScVal has 22 possible values since it’s the universal representation for a value inside of a smart contract. You can see all of them being inspected in the JavaScript utility scValToNative. Remember, XDR is the lowest level of communication on the Stellar network. Generally speaking, there are high-level abstractions that should help with these structures. For example, the above parsing routine is exactly the purpose of the Address.fromScAddress and Address.from_xdr_sc_address abstractions in the JavaScript and Python SDKs, respectively. In Go, ScAddress.String does the same job and returns the strkey. In Java, Address.fromSCAddress does the same job, and the address’s toString() returns the strkey. This greatly simplifies your code:

import { Address } from "@stellar/stellar-sdk";

// suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
const address = Address.fromScAddress(scAddr);

from stellar_sdk import Address

# suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
addr = Address.from_xdr_sc_address(scAddr)

// suppose `scAddr` is an instance of `xdr.ScAddress` we're parsing
address, err := scAddr.String() // G... for an account, C... for a contract

import org.stellar.sdk.Address;

// suppose `scAddr` is an instance of `org.stellar.sdk.xdr.SCAddress` we're parsing
Address address = Address.fromSCAddress(scAddr); // address.toString() is G... or C...

Always try to find a higher-level abstraction before digging into the raw structure.

Structures

Like in any C-like language, a structure is a packed object containing a bunch of fields of arbitrary types. Structures can themselves contain structures, so be sure you traverse the entire tree of fields when you’re constructing one. Let’s take, for example, the ContractEvent structure. Its definition is a little complicated, so let’s break it down:

An ExtensionPoint is a way for the protocol to be extended while maintaining backwards binary compatibility.

It’s basically an empty slot into which you can add variations in the future. The SorobanTransactionMeta structure is a great example of this in action: When the structure was initially added to the protocol, it just contained the details of an invocation (events and a return value). Later, we needed more details about the invocation in the resulting metadata, so SorobanTransactionMetaExt was injected into the ExtensionPoint with a V1 variant containing metadata about fees charged as a result of the invocation.

The contractID should be self-explanatory: it’s the hash of the contract ID that caused this event. It’s a pointer, though, because not all events are related to a specific contract. Some may come from the system, diagnostics, etc.

It should not be surprising that the field is stored as a Hash: even though at the ecosystem layer we refer to contracts by their “strkey” or string representation (in the form C...), these are actually “friendlier” representations of a raw SHA-256 hash. This is common at the protocol layer: the simplest, most compact representation will always be used. There are generally SDK tools to make translating these into more user-friendly types (as demonstrated in the example below).

The ContractType is essentially a way to differentiate which of the fields in the structure you should expect to be present and isn’t worth elaborating on.
Finally, we have the curious v union field.

You will often see unions in structures and vice-versa. If you have a handle on how each of these is traversed individually, you can handle them interleaved amongst each other. As a demonstration of how this is done with an SDK, even catered specifically to this structure, we can take a look at the source of the humanizeEvents helper method in the JavaScript SDK. Specifically, we’ll look at the details of the extractEvent subroutine, replicated here for convenience:

function extractEvent(event: ContractEvent): SorobanEvent {
  const contractId = event.contractId;
  const body = event.body.value;

  return {
    ...(contractId !== null &&
      contractId !== undefined && {
        contractId: StrKey.encodeContract(contractId.toBytes()),
      }),
    type: event.type.name,
    topics: body.topics.map((t: ScVal) => scValToNative(t)),

    data: scValToNative(body.data),
  };
}

In this short and sweet example we can see tons of the nuances we’ve discussed in this guide at work:

By checking whether the contractId is set (it’s a pointer, remember?), we decide whether or not we want to turn the raw hash into an ecosystem-compatible contract ID. toBytes() unwraps the raw bytes of the hash for the strkey encoder.
We access the .name field of the event type, an enum, which gives a human-readable string (such as "contract") rather than a number.
We inspect deeply into the body member to extract the topics and data fields separately, converting each of those opaque ScValues into JavaScript-friendly equivalents.

In the last part, we “abuse” the fact that there is only one variation of the body union we discussed: there is only the initial version (an arm with v == 0), so we can use the .value property to give us the underlying value directly rather than going through the rigamarole of differentiating the different cases. Reading .v0 would have been equivalent, but the benefit here is that if a v == 1 variation is introduced that also has a topics array and a data field (i.e. it adds additional fields rather than changing the v0 structure), this code will continue to work! There are many cases in which the different union arms share structure, and .value lets you take advantage of that.

Record Marking / Frames in Streams

In some applications like stellar-core, when multiple XDR objects are stored sequentially in a file or stream, each object is framed using the Record Marking Standard defined in RFC 5531 Section 11. Each record is composed of one or more fragments. Each fragment begins with a 4-byte header followed by the fragment data:

Bit 31 (high bit): Set to 1 if this is the last fragment of the record, 0 if more fragments follow.
Bits 0-30: The length in bytes of the fragment data that follows the header.

The header is encoded as a 4-byte big-endian unsigned integer. Note that this header is not itself in XDR standard form. In Stellar’s usage today each record contains exactly one XDR object, encoded as a single fragment with the last-fragment bit always set. This framing is used in:

History archives: Bucket files and checkpoint ledger files written and read by stellar-core.
Streaming LedgerCloseMeta: The LedgerCloseMeta stream output by stellar-core.

When decoding XDR with the stellar-cli, streams framed with the record marking can be decoded by specifying the --input stream-framed.

This overview should give you a strong baseline on understanding how to inspect XDR in your respective SDK to dig into the fields that you’re interested in. Specifics will, of course, be language dependent, but if you start at the foundation—that being the raw .x files themselves—you will get an understanding of the structure itself and should be able to access that same structure directly in your language of choice, as we’ve outlined here.