Clarification on continuous_index Handling Multi-Variable Inputs in CEBRA

[Barbod1380]

Hi, I’m working with the CEBRA AI package, specifically in the multi-session training setup. I have a question regarding the continuous_index parameter.
From the documentation, I understand that continuous_index is an auxiliary variable (e.g., head direction or position). However, I am unsure how CEBRA handles multi-variable inputs for continuous_index.

For example, if I pass a 2D numpy array with the shape (n, 3) as the continuous_index, does the model:

• Treat the three columns as separate features?
• Or, consider it as a single feature with three components?

I would appreciate any clarification on how continuous_index is processed in this scenario.
Thank you for your help!

[stes]

Hi @Barbod1380 , by default, all continuous features are processed together, this is done due to how distance are calculated here

CEBRA/cebra/distributions/index.py

Lines 76 to 99 in 5f46c32

	def __call__(self, query, mask=None):
	"""Compute the pairwise distances between index and query.
	Args:
	query: (n, d)
	The query matrix
	mask: (N,)
	A binary mask with same length as the index
	Returns: (M,n)
	Pairwise distances between the m samples with True binary
	mask (default: N) and the given query samples.
	"""
	# TODO(stes): slow
	query = query.to(self.device)
	qTq = query.square().sum(1, keepdim=True).T
	if mask is None:
	xTq = torch.einsum("ni,mi->nm", self.index, query)
	xTx = self.xTx
	else:
	xTq = torch.einsum("ni,mi->nm", self.index[mask], query)
	xTx = self.xTx[mask]
	return xTx + qTq - 2 * xTq

.

It would be possible to specify a component-wise handling based on your application example. For that

• you would implement a distance matrix with the metric of your choice
• you would use this distance matrix in the continuous sampling

Please let me know if that addresses your question!