Skip to main content
Entropy logoLink to Entropy
. 2026 May 29;28(6):611. doi: 10.3390/e28060611

Dialectics for Artificial Intelligence

Zhengmian Hu 1
Editor: Boris Ryabko1
PMCID: PMC13298821  PMID: 42352121

Abstract

Can artificial intelligence discover, from raw experience and without human supervision, concepts that humans have discovered? One challenge is that human concepts themselves are fluid: conceptual boundaries can shift, split, and merge as inquiry progresses (e.g., Pluto is no longer considered a planet). To make progress, we need a definition of “concept” that is not merely a dictionary label, but a structure that can be revised, compared, and aligned across agents. We propose an algorithmic information viewpoint that treats a concept as an information object defined only through its structural relation to an agent’s total experience. The core constraint is determination: a set of parts forms a reversible consistency relation if any missing part is recoverable from the others (up to the standard logarithmic slack in Kolmogorov complexity). This reversibility prevents “concepts” from floating free of experience and turns concept existence into a checkable structural claim. To judge whether a decomposition is natural, we define excess information, measuring the redundancy overhead introduced by splitting experience into multiple separately described parts. On top of these definitions, we formulate dialectics as an optimization dynamics: as new patches of information appear (or become contested), competing concepts bid to explain them via shorter conditional descriptions, driving systematic expansion, contraction, splitting, and merging. Finally, we formalize low-cost concept transmission and multi-agent alignment using small grounds that allow another agent to reconstruct the same concept under a shared protocol, making communication a concrete compute-bits trade-off.

Keywords: Kolmogorov complexity, algorithmic information theory, concept formation, concept transmission, multi-agent alignment, compression

1. Introduction

To build artificial intelligence, it is useful to look at human intelligence, not as an ideal to imitate in every detail, but as evidence that certain mechanisms are possible. Human concepts are both reusable and revisable. From finite and partial experience, humans form concepts such as “water”, “animal”, or “price” that can be applied to new cases to explain, predict, and guide action. Yet concepts can also change as inquiry progresses: boundaries may shift, cases may be reclassified, and previously separate notions may merge. In Ancient Greek astronomy, the “morning star” and the “evening star” were for a time treated as different objects and later unified as Venus. Similar merging and boundary revision recur throughout science.

If a concept is only a label attached to examples, the label alone cannot explain why experience does not remain a list of disconnected episodes, why independent agents with overlapping experience often reconstruct similar concepts, or why a few examples, typical features, or a short description can communicate a concept with a large extension. We therefore need a notion of concept that is tied to experience, supports revision, and makes communication cost explicit.

This paper develops such a notion using algorithmic information theory. We represent an agent’s relevant experience as a finite string I. Concepts are also finite strings, but a string counts as a concept only through its structural relation to I. The basic relation is a reversible consistency constraint called a determination: in a tuple of components, any missing component is recoverable from the remaining components, up to the standard logarithmic slack in Kolmogorov complexity. For the central case of a split (I,P1,P2), the pair (P1,P2) losslessly represents I, while each side is recoverable from the whole and the other side. Thus P1 and P2 are not independent names. Their identities are determined relative to the experience record I and the complementary side.

To compare such lossless decompositions, we use excess information: the extra description length paid when the whole is represented by separately described parts. Low-excess determinations provide natural decompositions, because they introduce little redundancy and little additional information beyond what is already present in the whole. In this sense, concept formation is not unconstrained naming, but a search for compact reversible decompositions of experience.

On top of this static structure, we define dialectics: an excess information optimization process over feasible determinations. Given a current split, a new or contested patch of information X may be absorbed by one side or the other. When the enlarged candidate concepts explicitly encode X and the corresponding determination constraints hold, the excess comparison reduces, up to logarithmic slack, to comparing the conditional description lengths of X under the competing concepts. The patch is assigned to the side that compresses it better under the current code. Repeated local moves can revise concept boundaries and reorganize decompositions when better codes are found.

The same framework accounts for low-cost concept transmission and multi-agent alignment. If two agents have sufficiently overlapping experience and share a public reconstruction protocol, the sender need not transmit the full extension of a concept. A small ground can pin down the intended side of a decomposition, and the receiver can reconstruct the concept’s extension inside its own experience record by running the shared dialectical procedure. Alignment becomes a concrete communication–computation trade-off governed by the same reversible constraints and code-length criterion.

Contributions

  • This paper gives a structural definition of concepts as components of determinations, so the claim that a string is a concept of I depends on a reversible information-theoretic condition rather than on a label alone.

  • Excess information provides a criterion for evaluating concept decompositions by measuring the redundancy cost of splitting in description length terms.

  • Dialectics is formulated as excess information optimization over feasible determinations, where concepts compete to explain new or contested information by shortening conditional descriptions, giving an operational account of concept growth and boundary revision.

  • This paper formalizes low-cost concept transmission and multi-agent alignment using small grounds that allow another agent to reconstruct the intended concept under overlapping experience and a shared reconstruction protocol.

Extended Kolmogorov background, detailed proofs, related work, practical algorithmic sketches, and speculative directions are provided in the Supplementary Materials.

2. Background on Kolmogorov Complexity

Kolmogorov complexity formalizes the idea that an object is simple when it has a short effective description. We work with finite binary strings and fix a universal prefix-free machine U. The prefix-free complexity K(x) is the length of the shortest program that outputs x on U. The conditional complexity K(x∣y) is the shortest description of x when y is given as side information. The joint complexity K(x,y) denotes the complexity of a fixed computable pairing of x and y. By the invariance theorem, changing U changes values only by an additive constant.

Throughout, =+ and ≤+ hide O(log(n+1)) terms, where n bounds the relevant complexities. We use Kraft’s inequality and its standard counting consequences, chain rules, symmetry of information, algorithmic mutual information

I(x;y)=K(x)+K(y)−K(x,y),

conditional mutual information, and data processing. Since K is uncomputable but upper semicomputable, exact optimality is generally unavailable. Later arguments therefore use explicit descriptions and computable upper bounds supplied by codes, probabilistic models, MDL-style criteria, or practical compressors [1,2,3,4,5,6]. A self-contained review of these background facts is given in Supplementary Material S3.

3. Algorithmic Parity Structures (Determinations)

Let x1:n=(x1,…,xn) be finite binary strings, with n≥2. For each i, write x−i for the tuple obtained by deleting xi.

Definition 1 

(Algorithmic parity structure/determination). An n-tuple x1:n is an algorithmic parity structure, or determination, relative to an optional context z, if every component is algorithmically recoverable from the remaining components:

∀i∈[n],K(xi∣x−i,z)=+0. (1)

Each component xi will be called a concept participating in this determination.

The name is meant to evoke parity. For bits, if xn=x1⊕…⊕xn−1, then any missing bit is determined by the others. Definition 1 replaces the XOR rule by arbitrary effective reconstruction, with reconstruction cost measured by conditional Kolmogorov complexity.

The central concept split used below is a 3-way determination (I,P1,P2). It states

K(I∣P1,P2)=+0,K(P1∣I,P2)=+0,K(P2∣I,P1)=+0.

Thus (P1,P2) is a lossless split representation of I, while each side is recoverable from the whole and the other side. Figure 1 depicts this three-way determination.

Figure 1.

Figure 1

A 3-way determination (I,P1,P2). The pair (P1,P2) losslessly represents I, and each component is recoverable from the other two.

In the binary-split setting used here, we call each side Pj a concept of I relative to this split. The qualifier “of I” is essential: Pj is not a concept by itself, but a component whose boundary and identity are fixed through a reversible relation with the whole experience I and the complementary side.

Finally, conditioning on a shared context z can be treated as ordinary participation in a larger determination: up to the same slack, an n-way determination relative to z can be viewed as an unconditional determination on (x1,…,xn,z,z), since each copy of z is recoverable from the other.

3.1. One-Variable Completion

Determinations can be used as constraints. Suppose x1,…,xn are fixed and we seek one additional string xn+1 that closes the tuple into a determination:

minxn+1K(xn+1)s.t.K(xn+1∣x1:n)=+0,∀i∈[n],K(xi∣x−i,xn+1)=+0. (2)

These constraints are exactly the requirement that (x1,…,xn,xn+1) forms a determination. The feasible set is nonempty: xn+1:=(x1,…,xn) is feasible. The optimal completion is a minimal piece of glue that makes all missing-coordinate reconstructions possible.

If xn+1 is feasible, then for every i, (x−i,xn+1) reconstructs xi; hence, K(xi∣x−i)≤+K(xn+1), so the largest missing-coordinate conditional complexity is a necessary cost. The theorem shows that this lower bound is tight up to logarithmic slack.

Theorem 1 

(One-variable completion cost). For any fixed n≥2 and strings x1:n, the optimal value of (2) is

maxi∈[n]K(xi∣x−i)

up to logarithmic slack.

The same completion problem relativizes to a fixed shared context z: keep the objective K(xn+1), replace each conditional complexity in the constraints by K(·∣·,z), and require feasibility to mean that (x1,…,xn,xn+1) forms a determination relative to z. Its optimal value is

maxi∈[n]K(xi∣x−i,z)

up to logarithmic slack.

This theorem shows that enforcing a reversible consistency relation has an explicit information cost. The fuller discussion of its connection to Muchnik-style multi-conditional descriptions and information distance is given in Supplementary Material S4, and the proof is collected in Supplementary Material S9 [7,8].

One-variable completion is the cleanest case: its optimal cost is the largest missing-coordinate conditional complexity. With two or more free components, completion becomes a joint optimization under determination constraints: information can be distributed across the free variables.

3.2. Pivoting and Concept Expansion

Determinations can be composed into networks. We next consider a basic local rewrite of such networks. Consider a two-node chain of three-way determinations sharing one bridge concept, for example (A,B,Z) and (Z,C,D). A pivot exists if there is another bridge F such that (A,C,F) and (B,D,F) are also determinations. Thus the same four strings A,B,C,D form one two-node chain through the original bridge Z and, if the pivot exists, another through the different bridge F. This second chain is an additional feasibility condition, not a consequence of the original chain.

A pivot bridge can sometimes be read as an expanded concept. In Figure 2, if D is interpreted as an existing concept and B as the information being absorbed, then the new bridge F in (B,D,F) can be read as D expanded by absorbing B. The competing absorptions in Figure 3 use the same two-node graph topology, with this expansion reading made explicit.

Figure 2.

Figure 2

A pivot for two adjacent 3-way determinations. The left diagram is the original chain with bridge Z, and the right diagram is the pivoted chain with bridge F.

Figure 3.

Figure 3

Competing absorptions of a patch. The contested patch X may be absorbed into P1 or into P2, producing two competing feasible splits.

Theorem 2 

(Pivot moves are not universal). There exist finite strings A,B,C,D,Z such that (A,B,Z) and (Z,C,D) are three-way determinations sharing the bridge concept Z, but there is no string F such that both triples (A,C,F) and (B,D,F) are determinations.

Thus pivoting is best viewed as an opportunity rather than a universal identity. When no pivot exists, the failure indicates dependence that cannot be re-expressed through a different bridge without adding new information. Supplementary Material S4 gives the finite-field counterexample and discusses its connection to extractable common information, while Supplementary Material S9 gives the rest of the proof, including the mixing lemma.

3.3. Excess Information

For a three-way determination π=(A,B,C), we read (B,C) as a split representation of A. Since A is recoverable from (B,C),

K(A)≤+K(B,C)≤+K(B)+K(C).

The overhead paid by describing B and C separately is

Δ(A,B,C):=K(B)+K(C)−K(A). (3)

We call this the excess information of the determination. By the preceding inequality, Δ(A,B,C)≥+0.

Excess information is the objective used here for evaluating lossless decompositions of a whole into parts. Since K(A∣B,C)=+0,

Δ(A,B,C)=+I(B;C)+K(B,C∣A),

so the same quantity records the duplicated information incurred when B and C are encoded separately and the additional information in (B,C) beyond A. Thus a low-excess decomposition avoids separating parts that share substantial mutual information and avoids introducing artifacts not present in the original information A.

For every fixed A and excess budget L, only finitely many pairs (B,C) make (A,B,C) a determination with Δ(A,B,C)≤L. Indeed, the excess bound implies

K(B,C∣A)≤+L,

so Kraft counting gives at most 2L+O(logK(A)) such pairs. Consequently, for fixed A and L, the object of study is a finite family of feasible decompositions. For fixed A, this bound scales exponentially with L, so very low-excess budgets leave a highly constrained feasible family.

4. Dialectics as an Optimization Problem

We now treat concept formation and revision as excess information optimization over feasible determinations. For a fixed experience string I, the simplest dialectical objective is

minP1,P2K(P1)+K(P2)−K(I)s.t.(I,P1,P2)isadetermination. (4)

A feasible pair (P1,P2) is a lossless split representation of I: the whole is recoverable from the two sides, and each side is recoverable from the whole together with the other side. The objective measures the extra cost of describing the two sides as separate concepts. Since I is fixed, minimizing excess over feasible decompositions is equivalent to comparing K(P1)+K(P2), the description length of representing the whole through its recoverable parts.

At the limiting ideal of unbounded computation, dovetailing over the universal description language makes the whole-object upper bound for I converge from above to K(I), though without a finite-time optimality certificate. At that limit, no feasible split can make K(P1)+K(P2) fall below K(I) except up to additive slack, while P1=I, P2=ϵ already gives O(1) excess. Thus, as the attempt to understand I is pushed further, boundaries between concepts tend to recede: relations among the concepts are integrated into a whole-object description of I, so that the concepts are subsumed by that whole-object description rather than kept as separate parts.

For finite-computation implementations, the relevant quantities are achieved description lengths under concrete compressors, probabilistic models, prequential codes, or MDL-style two-part codes [5,6,9]. Under such restricted coding protocols, a nontrivial decomposition can achieve a shorter code-length than a single whole-object code, because different sides can expose different regularities and be described by methods better matched to them. Dialectics therefore compares realizable decompositions by the tracked excess information upper bounds supplied by the available coding protocol. For the local patch moves below, this finite-computation comparison is between two feasible absorptions of the same patch X.

4.1. Local Patch Competition

A common local move relocates a small information patch X from one concept to another. Suppose the same information object I admits two feasible candidate splits: P1 absorbs X, producing a determination (I,P1′,P2), or P2 absorbs X, producing a determination (I,P1,P2′). Following the pivot constructions of Supplementary Material S4, assume also that the absorption triples (P1′,P1,X) and (P2′,P2,X) satisfy the corresponding determination constraints. Holding I fixed, the excess objective compares

K(P1′)+K(P2)againstK(P1)+K(P2′). (5)

The difference between these two objective values can be rewritten using conditional complexities of X. The reduction uses the following identity for three-way determinations.

Lemma 1 

(Complexity–difference identity). If (A,B,C) is a three-way determination, then

K(A)−K(B)=+K(C∣B)−K(C∣A).

Applying Lemma 1 to the two absorption triples and subtracting the resulting identities gives

K(P1′)+K(P2)−K(P1)+K(P2′)=+K(X∣P1)+K(X∣P2′)−K(X∣P1′)+K(X∣P2). (6)

If the enlarged candidate concepts P1′ and P2′ explicitly encode the absorbed patch X, then K(X∣P1′) and K(X∣P2′) are small. This is an additional coding assumption, not a consequence of determination alone. Under this assumption, the comparison is governed, up to logarithmic slack and the small terms K(X∣P1′) and K(X∣P2′), by

K(X∣P1)versusK(X∣P2).

In the idealized case where the small terms are O(1), the patch is assigned to the side that gives the shorter conditional description of X. Operationally, concepts compete to produce shorter descriptions of the same patch under their side information, and the better-compressing side gets to incorporate it.

The local rule gives a concrete instance of the zeroth-order procedure: generate the two feasible absorptions, compare computable upper bounds on their excess information, and retain the candidate with the smaller bound. If one accepts only moves that strictly decrease the current tracked upper bound, that bound is monotone and acts as a Lyapunov quantity for the optimization dynamics. This does not make the unknown true excess information monotone, nor make assignments irreversible: later improvements in coding methods may support reassigning a patch. Since the comparisons above hold only up to logarithmic slack, a practical margin rule is to commit only when the code-length gap is large relative to expected slack and estimation noise, near ties remain revisable.

4.2. Grounded Dialectics

The split (I,P1,P2) is symmetric: swapping the two sides preserves both feasibility and excess information. To refer to, re-identify, or communicate one side, we introduce grounds: asymmetric side-information strings A^,B^ that anchor the two sides.

For fixed I and grounds (A^,B^), define grounded dialectics by

minA,BK(A∣A^)+K(B∣B^)s.t.(I,A,B)isadetermination. (7)

The grounds identify the two sides, and the objective encourages each side to stay close to its ground by a conditional description penalty. Any target feasible split can be pinned down trivially by taking the grounds to be the two sides themselves. The useful case is when small grounds still identify a split. Any optimal solution of grounded dialectics satisfies the following excess bound, showing that small grounds can only select low-excess determinations.

Theorem 3 

(Ground complexity controls excess). Let (A★,B★) be optimal for (7). Then

K(A^)+K(B^)≥+K(A★)+K(B★)−K(I).

Grounded dialectics can be used as a communication mechanism between different agents. The setup requires sufficiently overlapping experience and a shared deterministic reconstruction protocol with fixed compute budget. The sender transmits grounds, and the receiver runs the protocol to reconstruct the intended side in its own experience record. Thus the sender need not transmit the full extension: a small ground can communicate a large concept.

4.3. Boundary-Mediated Splits

Many concrete segmentation-like problems include an explicit boundary object B: a curve, binary mask, set of polygon vertices, text span, or time-frequency mask. In this regime, B is the main decision variable: once I and B are fixed, P1,P2 are induced as algorithmic restrictions of I to the two sides. The standard boundary assumptions for this regime are

K(B∣Pi)=+0,K(Pi∣I,B)=+0(i=1,2),K(I∣P1,P2)=+0.

These assumptions say that the boundary is recoverable from either side, the parts are computable from the whole and the boundary, and the two parts cover the whole. Together they imply that (I,P1,P2) is a determination.

The split is scored by the excess information

Δ:=K(P1)+K(P2)−K(I).

Under the boundary assumptions, the same objective can be written as

Δ=+K(B)+I(P1;P2∣B)+K(B∣I).

This form separates three sources of overhead: boundary complexity, residual dependence between the two sides after the boundary is known, and ambiguity of the boundary given the whole. Other equivalent accounting forms, implication relations, and diagrams are deferred to Supplementary Material S5.

For a local boundary move, let X be a contested block near the boundary. Assume X lies outside both current sides. The two candidate assignments are

(P1′,P2)withboundaryB1,(P1,P2′)withboundaryB2.

Here Pi′ denotes Pi enlarged by including X. Let

Δ1=+K(P1′)+K(P2)−K(I),Δ2=+K(P1)+K(P2′)−K(I)

be the corresponding excess information values. Since I is fixed,

Δ1−Δ2=+K(P1′)−K(P1)−K(P2′)−K(P2).

Using the structured assumptions and derivation given in Supplementary Material S5, this becomes

Δ1−Δ2=+K(X∣P1)−K(X∣P2)+K(B1∣P2′)−K(B2∣P1′). (8)

The first bracket is the content bid: it favors assigning X to the side that compresses it better. The second bracket is the boundary bid: it penalizes the assignment that leaves a more complex or less visible boundary. When the boundary terms are negligible or comparable, the rule reduces to assigning X to the side with the shorter conditional code K(X∣Pi).

5. Positioning and Conclusions

The framework is closest in spirit to MDL and related coding approaches: good explanations are evaluated by the description lengths they achieve under an effective code [5,6]. It also relates to compression-based clustering and normalized information distance, where similarity is measured by shared description length [10,11]. The difference is that the present object is neither merely a pairwise distance nor a fixed model class, but a reversible decomposition constrained by determinations.

Under restricted computable code families, classical clustering, mixture modeling, and segmentation instantiate the same competitive-coding rule in different forms. In k-means, assigning a point to the nearest centroid is equivalent, up to constants, to choosing the cluster with the smallest squared-error coding surrogate [12,13]. In EM for mixture models, components compete through posterior responsibilities computed from mixture weights and likelihoods, while negative log-likelihood provides the corresponding code-length objective [14]. In boundary-based segmentation, regions compete by the joint cost of coding region content and boundary structure [15,16]. These methods do not use near-optimal Kolmogorov codes, but they instantiate the same competitive compression pattern. In this sense, dialectics does not replace these methods. It abstracts their shared structure by separating the objective, the computable code family used to upper-bound it, and the local move protocol used to compare feasible solutions.

We have described concepts as information objects tied to experience by reversible determinations. Excess information measures how expensive it is to represent a whole by separately described parts, and dialectics is the resulting excess information optimization dynamics: in local patch moves where the competing absorptions remain feasible and the enlarged concepts explicitly encode the contested patch, the comparison reduces, up to logarithmic slack, to conditional description lengths for that patch. Grounds provide an alignment mechanism: under overlapping experience and a shared reconstruction protocol, compact grounds can pin down intended concepts and support their reconstruction.

A natural next step is to scale compute for dialectics over larger experience records, so that previously unnamed concepts can be searched for as new low-excess determinations. Extended discussion is left to the Supplementary Materials.

Acknowledgments

During the preparation of this manuscript, the author used GPT series models for text editing and language polishing. The author reviewed and edited all AI-assisted output and takes full responsibility for the content of the publication.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/e28060611/s1, Supplementary Material S1, guide, S2, extended motivation, S3, Kolmogorov background, S4, determinations and pivot moves, S5, dialectical optimization and grounds, S6, related work, S7, practical methods, S8, future directions, S9, proofs, S10, limits of computable code-length upper bounds, S11, notation table, and S12, references. Additional references cited in the Supplementary Materials are included here [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47].

entropy-28-00611-s001.zip (470.5KB, zip)

Institutional Review Board Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study.

Conflicts of Interest

Author Zhengmian Hu was employed by Adobe Research, Adobe Inc. The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding Statement

This research received no external funding.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

References

  • 1.Kolmogorov A.N. Three approaches to the quantitative definition of information. Probl. Inf. Transm. 1965;1:1–7. doi: 10.1080/00207166808803030. [DOI] [Google Scholar]
  • 2.Solomonoff R.J. A formal theory of inductive inference. Part I. Inf. Control. 1964;7:1–22. doi: 10.1016/S0019-9958(64)90223-2. [DOI] [Google Scholar]
  • 3.Levin L.A. Laws of information conservation (non-growth) and aspects of the foundation of probability theory. Probl. Inf. Transm. 1974;10:206–210. [Google Scholar]
  • 4.Li M., Vitányi P.M.B. An Introduction to Kolmogorov Complexity and Its Applications. 4th ed. Springer; Cham, Switzerland: 2019. [Google Scholar]
  • 5.Rissanen J. Modeling by shortest data description. Automatica. 1978;14:465–471. doi: 10.1016/0005-1098(78)90005-5. [DOI] [Google Scholar]
  • 6.Grünwald P.D. The Minimum Description Length Principle. MIT Press; Cambridge, MA, USA: 2007. [Google Scholar]
  • 7.Muchnik A.A. Conditional complexity and codes. Theor. Comput. Sci. 2002;271:97–109. doi: 10.1016/S0304-3975(01)00033-0. [DOI] [Google Scholar]
  • 8.Vitányi P.M.B. Information distance in multiples. IEEE Trans. Inf. Theory. 2011;57:2451–2456. doi: 10.1109/TIT.2011.2110130. [DOI] [Google Scholar]
  • 9.Dawid A.P. Present position and potential developments: Some personal views: Statistical theory: The prequential approach. J. R. Stat. Soc. Ser. A (Gen.) 1984;147:278–292. doi: 10.2307/2981683. [DOI] [Google Scholar]
  • 10.Bennett C.H., Gács P., Li M., Vitányi P.M.B., Zurek W.H. Information distance. IEEE Trans. Inf. Theory. 1998;44:1407–1423. doi: 10.1109/18.681318. [DOI] [Google Scholar]
  • 11.Cilibrasi R., Vitányi P.M.B. Clustering by compression. IEEE Trans. Inf. Theory. 2005;51:1523–1545. doi: 10.1109/TIT.2005.844059. [DOI] [Google Scholar]
  • 12.MacQueen J. Some methods for classification and analysis of multivariate observations. In: Le Cam L.M., Neyman J., editors. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Volume 1. University of California Press; Downtown Oakland, CA, USA: 1967. pp. 281–297. [Google Scholar]
  • 13.Lloyd S. Least squares quantization in PCM. IEEE Trans. Inf. Theory. 1982;28:129–137. doi: 10.1109/TIT.1982.1056489. [DOI] [Google Scholar]
  • 14.Dempster A.P., Laird N.M., Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 1977;39:1–22. doi: 10.1111/j.2517-6161.1977.tb01600.x. [DOI] [Google Scholar]
  • 15.Zhu S.C., Yuille A.L. Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 1996;18:884–900. doi: 10.1109/34.537343. [DOI] [Google Scholar]
  • 16.Rao S.R., Mobahi H., Yang A.Y., Sastry S.S., Ma Y. Proceedings of the Asian Conference on Computer Vision (ACCV) Springer; Berlin/Heidelberg, Germany: 2009. Natural Image Segmentation with Adaptive Texture and Boundary Encoding; pp. 135–146. [Google Scholar]
  • 17.Shannon C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948;27:379–423, 623–656. doi: 10.1002/j.1538-7305.1948.tb01338.x. [DOI] [Google Scholar]
  • 18.Cover T.M., Thomas J.A. Elements of Information Theory. 2nd ed. Wiley; Hoboken, NJ, USA: 2006. [Google Scholar]
  • 19.Wallace C.S., Boulton D.M. An information measure for classification. Comput. J. 1968;11:185–194. doi: 10.1093/comjnl/11.2.185. [DOI] [Google Scholar]
  • 20.Ma Y., Derksen H., Hong W., Wright J. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE Trans. Pattern Anal. Mach. Intell. 2007;29:1546–1562. doi: 10.1109/TPAMI.2007.1085. [DOI] [PubMed] [Google Scholar]
  • 21.McLachlan G., Peel D. Finite Mixture Models. Wiley; Hoboken, NJ, USA: 2000. [Google Scholar]
  • 22.Blei D.M., Ng A.Y., Jordan M.I. Latent Dirichlet Allocation. J. Mach. Learn. Res. 2003;3:993–1022. [Google Scholar]
  • 23.Greff K., Srivastava R.K., Schmidhuber J. Binding via Reconstruction Clustering. arXiv. 2015 doi: 10.48550/arXiv.1511.06418.1511.06418 [DOI] [Google Scholar]
  • 24.Yang Y., Loquercio A., Scaramuzza D., Soatto S. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019. IEEE; New York, NY, USA: 2019. Unsupervised Moving Object Detection via Contextual Information Separation; pp. 879–888. [DOI] [Google Scholar]
  • 25.Savarese P., Kim S.S.Y., Maire M., Shakhnarovich G., McAllester D. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual, 19–25 June 2021. IEEE; New York, NY, USA: 2021. Information-Theoretic Segmentation by Inpainting Error Maximization; pp. 4029–4039. [DOI] [Google Scholar]
  • 26.Schmidhuber J. Learning Factorial Codes by Predictability Minimization. Neural Comput. 1992;4:863–879. doi: 10.1162/neco.1992.4.6.863. [DOI] [Google Scholar]
  • 27.Locatello F., Bauer S., Lucic M., Rätsch G., Gelly S., Schölkopf B., Bachem O. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations; Proceedings of the 36th International Conference on Machine Learning (ICML); Long Beach, CA, USA. 9–15 June 2019; pp. 4114–4124. [Google Scholar]
  • 28.Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative Adversarial Nets; Proceedings of the Advances in Neural Information Processing Systems; Montreal, QC, Canada. 8–13 December 2014; pp. 2672–2680. [Google Scholar]
  • 29.Adipoetra M., Martin S. Deep Generative Clustering with VAEs and Expectation-Maximization. arXiv. 2025 doi: 10.48550/arXiv.2501.07358.2501.07358v1 [DOI] [Google Scholar]
  • 30.Abdali S., Goksen C., Amizadeh S., Koishida K. Self-Reflecting Large Language Models: A Hegelian Dialectical Approach. arXiv. 2025 doi: 10.48550/arXiv.2501.14917.2501.14917 [DOI] [Google Scholar]
  • 31.Anghel C., Anghel A.A., Pecheanu E., Susnea I., Cocu A., Istrate A. Multi-Model Dialectical Evaluation of LLM Reasoning Chains: A Structured Framework with Dual Scoring Agents. Informatics. 2025;12:76. doi: 10.3390/informatics12030076. [DOI] [Google Scholar]
  • 32.Baxter R.A., Oliver J.J. Proceedings of the Algorithmic Learning Theory: 7th International Workshop, ALT ’96, Sydney, Australia, 23–25 October 1996. Volume 1160. Springer; Berlin/Heidelberg, Germany: 1996. The Kindest Cut: Minimum Message Length Segmentation; pp. 83–90. Lecture Notes in Computer Science. [DOI] [Google Scholar]
  • 33.Oliver J.J., Baxter R.A., Wallace C.S. Minimum Message Length Segmentation. In: Wu X., Kotagiri R., Korb K.B., editors. Proceedings of the Research and Development in Knowledge Discovery and Data Mining: Second Pacific-Asia Conference, PAKDD ’98, Melbourne, Australia, 15–17 April 1998; . Berlin/Heidelberg, Germany: Springer; 1998. pp. 222–233. Lecture Notes in Computer Science. [DOI] [Google Scholar]
  • 34.Bohlin L., Edler D., Lancichinetti A., Rosvall M. Community Detection and Visualization of Networks with the Map Equation Framework. In: Ding Y., Rousseau R., Wolfram D., editors. Measuring Scholarly Impact: Theory and Practice. Springer International Publishing; Cham, Switzerland: 2014. pp. 3–34. [DOI] [Google Scholar]
  • 35.Bommasani R., Hudson D.A., Adeli E., Altman R., Arora S., von Arx S., Bernstein M.S., Bohg J., Bosselut A., Brunskill E., et al. On the Opportunities and Risks of Foundation Models. arXiv. 2021 doi: 10.48550/arXiv.2108.07258.2108.07258 [DOI] [Google Scholar]
  • 36.Kirillov A., Mintun E., Ravi N., Mao H., Rolland C., Gustafson L., Xiao T., Whitehead S., Berg A.C., Lo W.Y., et al. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 1–6 October 2023. IEEE; New York, NY, USA: 2023. Segment Anything; pp. 4015–4026. [DOI] [Google Scholar]
  • 37.Zhang C., Han D., Qiao Y., Kim J.U., Bae S.H., Lee S., Hong C.S. Faster Segment Anything: Towards Lightweight SAM for Mobile Applications. arXiv. 2023 doi: 10.48550/arXiv.2306.14289.2306.14289 [DOI] [Google Scholar]
  • 38.Ke L., Ye M., Danelljan M., Liu Y., Tai Y.W., Tang C.K., Yu F. Proceedings of the Advances in Neural Information Processing Systems 36 (NeurIPS 2023), New Orleans, LA, USA, 10–16 December 2023. Curran Associates, Inc.; Red Hook, NY, USA: Segment Anything in High Quality; pp. 29914–29934. [Google Scholar]
  • 39.Hu E.J., Shen Y., Wallis P., Allen-Zhu Z., Li Y., Wang S., Wang L., Chen W. LoRA: Low-Rank Adaptation of Large Language Models; Proceedings of the International Conference on Learning Representations (ICLR); Virtual. 25–29 April 2022. [Google Scholar]
  • 40.Gács P., Körner J. Common Information Is Far Less Than Mutual Information. Probl. Control Inf. Theory. 1973;2:149–162. [Google Scholar]
  • 41.Romashchenko A. Pairs of Words with Nonmaterializable Mutual Information. Probl. Inf. Transm. 2000;36:1–18. [Google Scholar]
  • 42.Caillat-Grenier G., Romashchenko A., Zyavgarov R. Proceedings of the 2024 IEEE Information Theory Workshop (ITW), Shenzhen, China, 24–28 November 2024. IEEE; New York, NY, USA: 2024. Common information in well-mixing graphs and applications to information-theoretic cryptography; pp. 181–186. [DOI] [Google Scholar]
  • 43.Merchant A., Batzner S., Schoenholz S.S., Aykol M., Cheon G., Cubuk E.D. Scaling Deep Learning for Materials Discovery. Nature. 2023;624:80–85. doi: 10.1038/s41586-023-06735-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Zheng Y., Xu H., Li Z., Li L., Yu Y., Jiang P., Shi Y., Zhang J., Huang Y., Luo Q., et al. Artificial Intelligence-Driven Approaches in Semiconductor Research. Adv. Mater. 2025;37:2504378. doi: 10.1002/adma.202504378. [DOI] [PubMed] [Google Scholar]
  • 45.Brbić M., Zitnik M., Wang S., Pisco A.O., Altman R.B., Darmanis S., Leskovec J. MARS: Discovering Novel Cell Types across Heterogeneous Single-Cell Experiments. Nat. Methods. 2020;17:1200–1206. doi: 10.1038/s41592-020-00979-3. [DOI] [PubMed] [Google Scholar]
  • 46.Iwaniec H., Kowalski E. Analytic Number Theory. Volume 53 American Mathematical Society Colloquium Publications; Providence, RI, USA: 2004. [Google Scholar]
  • 47.Hoory S., Linial N., Wigderson A. Expander graphs and their applications. Bull. Am. Math. Soc. 2006;43:439–561. doi: 10.1090/S0273-0979-06-01126-8. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

entropy-28-00611-s001.zip (470.5KB, zip)

Data Availability Statement

No new data were created or analyzed in this study.


Articles from Entropy are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES