Kolmogorov complexity is uncomputable
Kolmogorov complexity
Kolmogorov complexity is uncomputable
Kolmogorov complexity measures the shortest program needed to produce an object, reflecting its computational resource requirements. This concept generalizes classical information theory and is named after Andrey Kolmogorov.
The notion of Kolmogorov complexity allows for stating and proving impossibility results similar to Cantor's diagonal argument, Gödel's incompleteness theorem, and Turing's halting problem. Specifically, no program can compute a lower bound for each text's Kolmogorov complexity that returns a value larger than the program's own length.
Example
Consider a simple text "hello". A shorter program might be "print 'hello'", while a longer program could be a detailed explanation of how to print "hello". The Kolmogorov complexity of "hello" is lower for the shorter program.
Understanding the uncomputability of Kolmogorov complexity is crucial for grasping the limitations of computational systems and the inherent unpredictability in certain computational tasks.
Related concepts
Halting problem
Alan Turing proved the halting problem is undecidable
Overlapping subproblems
Dynamic programming solves overlapping subproblems by storing results of subproblems to avoid redundant calculations
merge sort: O(n log n) always
Merge sort consistently performs at O(n log n) time complexity for any input size
Graph (abstract data type)
Time complexity of BFS and DFS: O(V + E)
Computational complexity of matrix multiplication
O(n³) naive matrix multiplication
Attention Is All You Need
O(n) complexity for long sequences
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews