A Split-and-Recombine Approach for Follow-up Query Analysis
Table 2: Variant results on FollowUp dev set.
['[BOLD] Variant', '[BOLD] SymAcc (%)', '[BOLD] BLEU (%)']
Besides baselines, we also conduct experiments with several variants of StAR to further validate the design of our model. The SymAcc drops from about 55% to 40% by ablating Phase I, and to 23% by ablating Phase II. Their poor performances indicate both of the two phases are indispensable. “– RL” also performs worse, which again demonstrates the rationality of applying RL. “+ Basic Reward” represents the most straightforward REINFORCE algorithm, which samples both q∈Q and ~z∈Z, then takes r(z,~z) as R(q,z). “+ Oracle Reward” assumes the conflicts are always correct and rewrites R(q,z) as max~z∈Z(r(z,~z)). “+ Uniform Reward” assigns the same probability to all ~z and obtains R(q,z) as mean(∑~z∈Zr(z,~z)) . StAR shrinks the space from |Q|⋅|Z| down to |Q| by enumerating ~z. Meanwhile, statistics show that StAR obtains a 15× speedup over “+ Basic Reward” on the convergence time.