Tensor Is the Might

srean•about 1 hour ago

If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour.

https://web.archive.org/web/20111015133833/http://www-staff....

This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.

[Programming in FISh] https://link.springer.com/article/10.1007/s100090050037

[Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D...

zserge•about 1 hour ago

I wish we used something else but Python as the default ML language, so that some advanced type system would work for us to ensure tensor shapes and compatibility.

bssrdf•32 minutes ago

I found [ggml] is really easy, educational and fun to use. It is written in C and the backbone for llama.cpp.

[ggml]https://github.com/ggml-org/ggml

churchsub1•44 minutes ago

Here's a Python library for static type checking of tensor shapes: https://pypie.dev/

srean•41 minutes ago

Thanks. Did not know about it.

Looks a little FiShy. In a good way [0].

[0] https://link.springer.com/article/10.1007/s100090050037

srean•about 1 hour ago

Honestly, I quite like Python most of the time. The times when I don't, argghh type errors.

The worst I have had in a while is when I was dealing with a sparse array data type and getting some processing done in a C function to speed things up.

Infuriatingly enough, the index type used to change after initialising the array. It would check if Int32 could accomodate the size of the array. If it could it would downgrade to Int32 at runtime. My C counterpart had no clue.

That was so not fun to debug because the errors were intermittent (depending on the fill factor of the sparse array)

srean•about 2 hours ago

> A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object.

Erm... many would disagree. I think what he means is just a multidimensional array.

zserge•about 2 hours ago

Not in a general mathematical sense, of course. Rather carthesian tensors. Seems like for most libraries it's `numpy.ndarray`: https://docs.tinygrad.org/tensor/ or https://docs.pytorch.org/docs/main/tensors.html

srean•about 1 hour ago

To add to the confusion, Mathematicians and Physicists often disagree on what that word means. A Physicist's tensor is often a mathematicians tensor field.

An appropriate tensor can certainly be represented by a multidimensional array but not all multidimensional arrays are tensors. It depends on how that multidimensional array participates with others.

For example, (lat, long) is not a 2d vector although it can be represented as a 2 dimensional array.

khalic•22 minutes ago

Why does does the diagram say tensors are 3D?

zserge•14 minutes ago

I guess I miscommunicated it. I meant I'd call everything a tensor but for some smaller dimensions we have specific names. Of course, 1D can still be a tensor and so is 6D or nD.

Tensor Is the Might

⚡ Community Insights

Discussion (11 Comments)Read Original on HackerNews