An advantage of a tokenised architecture is that it can accelerate the evaluation of queries. If a query initially expressed in terms of string pattern matches, is compressed, its new expression is in terms of integer comparisons. These are inherently faster as, the amount of data that has to be moved into the CPU and compared for each selection operation is significantly reduced.
Clearly the final answer has to be converted from the abstract to the concrete representation. This involves a decompression phase in which the field codes are mapped to their associated keywords. But the computational cost of this will be borne only on the tuples that are returned as a result. Since this phase is anyway dominated by output delays in the screen handling software, the perceived cost of this decompression is likely to be minimal.