feat(encryption): [2/N] Add standard key metadata

[xanderbailey]

Python version of StandardKeyMetadata direct port of the logic added in https://github.com/apache/iceberg-rust/pull/2340/changes

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

[kevinjqliu]

LGTM

i pushed a change to redact encryption_key in repr
6f6ceaa

[kevinjqliu]

BTW we want to rip out the current pyiceberg avro reader (which is written with cython) and replace with rust's avro reader. maybe this would be a good integration point 😄

[kevinjqliu]

just spitballing, maybe something like

@dataclass(frozen=True)
class StandardKeyMetadata:
    encryption_key: bytes = field(repr=False)
    aad_prefix: bytes | None = None
    file_length: int | None = None

    @classmethod
    def from_bytes(cls, data: bytes) -> StandardKeyMetadata:
        key, aad, length = pyiceberg_core.encryption.decode_standard_key_metadata(data)
        return cls(key, aad, length)

    def to_bytes(self) -> bytes:
        return pyiceberg_core.encryption.encode_standard_key_metadata(
            self.encryption_key, self.aad_prefix, self.file_length
        )

and in rust side:

// iceberg-rust/bindings/python/src/encryption.rs
#[pyfunction]
fn decode_standard_key_metadata(data: &[u8])
    -> PyResult<(Vec<u8>, Option<Vec<u8>>, Option<i64>)>
{
    let metadata = StandardKeyMetadata::from_bytes(data).map_err(to_py_err)?;
    Ok((metadata.key_bytes(), metadata.aad_prefix(), metadata.file_length()))
}

#[pyfunction]
fn encode_standard_key_metadata(
    key: &[u8],
    aad_prefix: Option<&[u8]>,
    file_length: Option<i64>,
) -> PyResult<Vec<u8>> {
    StandardKeyMetadata::new(key, aad_prefix, file_length)
        .and_then(|metadata| metadata.to_bytes())
        .map_err(to_py_err)
}

[xanderbailey]

This is pretty cool, let me have a look at this on Monday!

[kevinjqliu]

cheers, im pretty excited about this. we can brainstorm and write it up as an issue.

[xanderbailey]

Okay gave it a go here apache/iceberg-rust#3206 let me know what you think!