Model Sets with Permutation-Invariant Pooling

Fri, 11 Sep 2026 00:00:00 +0700

Model Sets with Permutation-Invariant Pooling

Many model inputs are collections rather than sequences. A shopping basket contains products, a scene contains detected objects, and a batch of sensor readings may represent simultaneous observations. Reordering those elements should not change a prediction that depends only on the collection.

A sequence model can still consume such data, but it may treat arbitrary input order as information. Permutation-invariant pooling gives the architecture the symmetry the task actually requires: encode each element with the same function, combine the encoded elements using an order-independent operation, then make the final prediction from that combined representation.

Use Cosine Classifiers to Separate Direction from Feature Magnitude

Fri, 11 Sep 2026 00:00:00 +0700

Use Cosine Classifiers to Separate Direction from Feature Magnitude

A standard neural classifier usually ends with a linear layer. That head scores each class with a dot product between the model’s feature vector and a class weight vector. The dot product mixes two signals: the angle between the vectors and their magnitudes.

A cosine classifier removes the magnitude part before scoring. It normalizes both vectors to unit length, then compares their directions. An explicit scale converts the resulting cosine similarities into logits suitable for a classification loss.

Representation on Nalar

Model Sets with Permutation-Invariant Pooling

Model Sets with Permutation-Invariant Pooling

Use Cosine Classifiers to Separate Direction from Feature Magnitude

Use Cosine Classifiers to Separate Direction from Feature Magnitude