I think narrow distilled LMs are really hard to get to do anything interesting -- 700GB at 32bits. But a 30B parameter model at 16 bits would be 60GB, small enough to fit on upper-end workstations. A well trained 30B parameter model w