Universal information: How much is enough?

A thought (it is probably as old as hills, but new to me):

Physics advances by finding rules our observations obey. A good theory reduces the information needed to describe what has been observed.
In each (most?) advance(s) we find that the universe actually contains (can be defined by) less information than we had previously (implicitly) assumed. When this is false, science is complete.

If we take that (that the universe contains much less information than we think) as a sort of meta law, then we could have expected a quantized physics would be required, to limit information ...

It amused me anyway!

Given the huge dataset of our observations, (much of which is redundant), what is the minimum set of equations needed to describe that dataset ?
How do we know it is the minimum ?
Why do we need to keep making observations ?

From a commercial point of view, the reasons for making high energy particle observations seem to be becoming small :-) - basically "they thought it was all finished in 1900 and look what happened".

Humans - the information animals - have dominated the world by finding and exploiting good models (understanding) of what is going on, so we are not going to stop trying any time soon.

It seems to be in our DNA.