Export
Hijax
[1]
[2]
0.85250294 0.8165137 0.7814907
Hijax Variable
State propagation:
[3]
0 1
[4]
{ lambda ; a:Variable(). let
jit[
name=inc
jaxpr={ lambda ; a:Variable(). let
b:i32[] = get_variable[
avals=(ShapedArray(int32[], weak_type=True),)
has_qdd=True
treedef=PyTreeDef(CustomNode(Variable[(('eager_sharding', True), ('hijax', True), ('mutable', True), ('ref', False))], [*]))
var_type=<class 'flax.nnx.variablelib.Variable'>
] a
c:i32[] = add b 1:i32[]
_:i32[] = get_variable[
avals=(ShapedArray(int32[], weak_type=True),)
has_qdd=True
treedef=PyTreeDef(CustomNode(Variable[(('eager_sharding', True), ('hijax', True), ('mutable', True), ('ref', False))], [*]))
var_type=<class 'flax.nnx.variablelib.Variable'>
] a
set_variable[
treedef=PyTreeDef(CustomNode(Variable[(('eager_sharding', True), ('hijax', True), ('mutable', True), ('ref', False))], [*]))
var_type=<class 'flax.nnx.variablelib.Variable'>
] a c
in () }
] a
in () }
Pytree values:
[5]
Variable( # 2 (8 B) value={'a': Array(0, dtype=int32, weak_type=True), 'b': Array(2, dtype=int32, weak_type=True)}, hijax=True ) Variable( # 2 (8 B) value={'a': Array(1, dtype=int32, weak_type=True), 'b': Array(4, dtype=int32, weak_type=True)}, hijax=True )
Dynamic state structure:
[6]
Before: Variable( value={}, hijax=True ) After: Variable( # 1 (4 B) value={'y_mean': Array(-1.1782329, dtype=float32)}, hijax=True )
[7]
Variable( # 3 (12 B) value=Array([1, 2, 3], dtype=int32), hijax=True )
Mutability
[8]
nnx.vars_as(model, mutable=False) = Linear( # Param: 3 (12 B) kernel=Param( # 3 (12 B) value=Array(shape=(1, 3), dtype=dtype('float32')), hijax=True, mutable=False ) ) nnx.vars_as(model, mutable=True) = Linear( # Param: 3 (12 B) kernel=Param( # 3 (12 B) value=Array(shape=(1, 3), dtype=dtype('float32')), hijax=True ) )
[9]
ImmutableVariableError: Cannot mutate Variable as it is marked as immutable. (https://flax.readthedocs.io/en/latest/api_reference/flax.errors.html#flax.errors.ImmutableVariableError)
Ref support
[10]
Variable( # 1 (4 B) value=Array(0, dtype=int32, weak_type=True), hijax=True, ref=True ) Ref(0, dtype=int32, weak_type=True)
[11]
immutable = Variable( # 1 (4 B) value=Array(0, dtype=int32, weak_type=True), had_ref=True, hijax=True, mutable=False ) mutable = Variable( # 1 (4 B) value=Array(0, dtype=int32, weak_type=True), hijax=True, ref=True )
Examples
[12]
Training Loop
[13]
loss = 1.000178 loss = 0.9700456 loss = 0.93967044
Scan Over Layers
[14]
Limitations
Mutable Outputs
[15]
Error: mutable hitypes should use lo_ty_qdd instead
[16]
model.linear = Linear( # Param: 128 (512 B) kernel=Param( # 128 (512 B) value=Array(shape=(2, 64), dtype=dtype('float32')), hijax=True ) )
Reference Sharing (aliasing)
[17]
None
[18]
None HasShared( # Variable: 1 (4 B) a=Variable( # 1 (4 B) value=Array(0, dtype=int32, weak_type=True), hijax=True ), b=Variable( # 1 (4 B) value=Array(0, dtype=int32, weak_type=True), hijax=True ) )
[19]
Duplicates found:
- [('a',), ('b',)]
[20]
HasShared( # Variable: 1 (4 B) a=Variable( # 1 (4 B) value=Array(5, dtype=int32, weak_type=True), hijax=True ), b=Variable( # 1 (4 B) value=Array(5, dtype=int32, weak_type=True), hijax=True ) )
[21]