To explain this rather fancy word, a heuristic is basically a ‘general rule of thumb’, being a shortcut for solving problems via the method of trial and error. On the face of it, heuristics are quite ugly and inefficient in solving problems. Thus, a heuristic is very much the antithesis of a mathematical proof – which often is elegant and even beautiful.

Essentially, heuristics are generated via a method of induction. We use heuristics when we have seen a similar problem/observation before, and so in that similar case, we can make the shortcut to say that we will solve it in a similar way as that previous scenario. Generally, the more cases that you have, the better the heuristic. The more brute force is applied to get more cases, the more likely the theory is to hold. In the context of the workforce for example, heuristics would be analogous to one’s experience in a particular field.

In many regards, the scientific method is like heuristics, albeit in a more controlled or pure manner. The key to all of science is the observation of phenomena that are isolated in such a way that only one variable changes, and then analysing the result. Heuristics are essentially the ugly observation of many variables and thus coming up with a ‘general principle’ because the scientific method won’t be able to apply as one won’t be able to isolate the system. Heuristics are thus invaluable in solving the set of problems that cannot be solved deductively. (Eddie Woo gives quite a nice explanation)

As an illustration, let’s just take the game of chess for example. In many ways, chess openings are heuristics. From the onset, the number of possible moves increase exponentially, where it is estimated that there are 10^120 possible games of chess – being known as the Shannon number. As a point of reference, there are 10^97 elementary particles in the universe. (These are particles that are smaller than atoms which include quarks, antiquarks, bosons etc.) In this way, it is impossible to deductively or computationally find what is the best move to play. This is where heuristics come in, in which via millions of games, humans have come up with opening lines that are generally conducive towards winning.

To elicit the notion of heuristics, I’ll compare how chess engines to how humans think. Even though computer engines have beaten human players in chess via the sheer brute force of computational power in calculating moves, humans think in a fundamentally different way to engines. Engines check for every iteration in which their computational power allows to compute several moves ahead, and it selects the best outcome on a material basis. (Engines technically check more then this, but I’ll keep it simple). Humans are unable to evaluate a position in a quantitative basis, however they do so in a more conceptual and qualitative manner. For instance, if you were to play a game, you would evaluate your position via the following but not exhaustive list:

• Do you have more pieces on a material basis than your opponent?
• Are your pieces better developed than your opponent, and did you do it more efficiently than them?
• Are your pieces more active than your opponent? i.e. is your light squared bishop more active than their light squared bishop.
• Does the development of your pieces restrict the ability of your opponent’s pieces?
{"found_math": false, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "url": "bryanhung.com", "fetch_time": 1714029044000000000, "content_mime_type": "text/html", "warc_filename": "crawl-data/CC-MAIN-2024-18/segments/1712297290384.96/warc/CC-MAIN-20240425063334-20240425093334-00562.warc.gz", "warc_record_offset": 127679182, "warc_record_length": 12967, "token_count": 688, "char_count": 3360, "score": 3.640625, "int_score": 4, "crawl": "CC-MAIN-2024-18", "snapshot_type": "longest", "language": "en", "language_score": 0.9439743757247925, "file_path": "/sailhome/yjruan/project/latent-lm/data/processed/finemath-4plus/finemath-4plus/train-00039-of-00064.parquet"}