WorldCup Sampling for Multi-bit LLM Watermarking Yidan Wang, Yubing Ren, Yanan Cao, Li Guo Abstract As large language models (LLMs) generate increasingly human-like text, watermarking offers a promising solution for reliable attribution beyond mere detection. While multi-bit watermarking enables richer provenance encoding, existing methods largely extend zero-bit schemes through seed-driven steering, leading to indirect information flow, limited effective capacity, and suboptimal decoding. In this paper, we propose WorldCup, a multi-bit watermarking framework for LLMs that treats sampling as a natural communication channel and embeds message bits directly into token selection via a hierarchical competition mechanism guided by complementary signals. Moreover, WorldCup further adopts entropy-aware modulation to preserve generation quality and supports robust message recovery through confidence-aware decoding. Comprehensive experiments show that WorldCup achieves a strong balance across capacity, detectability, robustness, text quality, and decoding efficiency, consistently outperforming prior baselines and laying a solid foundation for future LLM watermarking studies. 1 Introduction Large language models (LLMs) ( 22 ; 12 ; 2 ) have shown remarkable performance across a wide range of real-world applications, including creative writing, code generation and AI agent ( 41 ) , rendering LLM-generated text progressively indistinguishable from human-written text ( 44 ) . While these advances significantly boost productivity, they also amplify serious risks such as misinformation dissemination, academic plagiarism, and phishing attacks ( 50 ) . As a consequence, LLM watermarking has emerged as a promising approach to mitigate these concerns by embedding imperceptible yet verifiable signals into generated text, thereby enabling reliable attribution, detection and traceability ( 4 ) . Inference-time LLM watermarking can be broadly categorized into zero-bit and multi-bit based on embedding capacity ( 38 ) . While zero-bit watermarking focuses solely on detection, multi-bit watermarking further enables the extraction of rich metadata, such as model identity or generation timestamps. The prevailing multi-bit watermarking paradigm builds upon zero-bit schemes by implicitly embedding message into the stochastic generation process. Typically, a pseudo-random seed is derived by hashing a predefined secret key with the context and is associated with a target bit string to steer an underlying zero-bit watermarking mechanism, thereby establishing a verifiable link between the message and the generated text. Decoding reverses this process by searching over candidate messages to identify the one that best aligns the expected watermark signal. However, this steering-based construction also exposes a deeper limitation of the current paradigm. Rather than writing the message directly into the generative process, existing methods route information through a multi-stage pathway: the message is first mapped to a seed, the seed induces a pseudo-random perturbation pattern, and only then does the perturbation manifest indirectly in the generated text. Such a long information chain inevitably incurs entropy loss at each stage, since the watermark signal must survive multiple transformations before reaching the surface form. In effect, the sampling process is treated as a binary detection probe rather than as a native communication channel, which restricts the usable capacity of each token. These constraints reflect the inherent difficulty of forcing a zero-bit mechanism to shoulder a multi-bit objective. To overcome these limitations, we revisit multi-bit watermarking from a more direct and principled perspective. Rather than encoding messages indirectly through seed-driven steering, we treat the sampling step itself as a natural communication channel and embed bits directly into token selection. This shift is lightly motivated by Google’s SynthID Text ( 13 ) , which demonstrates that token-ranking perturbations can leave statistically verifiable traces without degrading text quality, albeit in a detection-only setting. Building on this insight, we extend the idea to a full-fledged multi-bit regime and introduce a multi-round, hierarchical sampling strategy that couples message embedding with the competitive selection of candidate tokens. Specifically, we propose WorldCup , a multi-bit watermarking framework for LLMs that leverages the inherent redundancy of autoregressive token generation to support robust and scalable information embedding. At the encoding stage, WorldCup conceptualizes watermarking as a structured competition process, in which multiple complementary signals jointly guide token selection and induce stable statistical separation, while preserving sufficient flexibility for high-quality natural language generation. To balance detectability and fluency, the framework further incorporates an entropy-aware modulation mechanism that adaptively adjusts watermark strength in response to local uncertainty in the model output distribution. At the decoding stage, WorldCup moves beyond simple counting-based detection and adopts a confidence-aware aggregation strategy that weights token-level evidence by its statistical reliability, thereby mitigating the disproportionate influence of low-entropy tokens. These design choices enable fine-grained control over the divergence between clean and watermarked distributions, and become increasingly advantageous as the embedded payload grows, yielding substantial improvements in decoding accuracy and efficiency at scale. To this end, comprehensive experiments across multiple LLMs and downstream tasks demonstrate that WorldCup consistently outperforms prior baselines, delivering a strong and well-balanced trade-off among multi-bit capacity, watermark detectability, robustness, text quality, and decoding efficiency. In summary, our contributions are threefold: • We propose WorldCup, a versatile multi-bit watermarking framework that is rigorously validated through both theoretical analysis and extensive empirical evaluation. • We introduce a confidence-aware decoding paradigm that moves beyond prior counting-based detectors, substantially improving both decoding accuracy and efficiency. • We conduct a systematic analysis of key design components, examining different hyperparameter choices and offering actionable insights to guide future research. 2 Preliminaries Notations. Consider an autoregressive LLM \Theta based on transformer ( 53 ) , let \mathcal{V} denote the vocabulary set of all tokens with size |\mathcal{V}| . Given a prompt p_{0} and previously contextual tokens \mathbf{x}_{2 groups of g -value functions, allowing each token to embed k bits. Detailed formulations are provided in Appendix G.3 . 3.4 Confidence-aware Message Decoding In contrast to prior counting-based decoding, our method employs a confidence-aware strategy that aggregates fine-grained token-level scores across groups associated with the same message position, rather than making hard binary decisions per token. This design yields substantially more stable detection statistics, as illustrated in Fig. 3 . Problem setup. Let \mathbf{y}_{1:T}=(y_{1},y_{2},\ldots,y_{T}) denote a generated text sequence of length T . As in the encoding stage, we consider k independent groups of g -value function families \{\mathbf{G}_{j}=(\mathbf{g}_{j},\mathbf{\bar{g}}_{j})\}_{j=1}^{k} . These k families jointly encode a k -bit message string at each token position. Accordingly, a binary message \{0,1\}^{b} is reparameterized as a sequence of 2^{k} -ary message \mathbf{m}=(m_{0},\ldots,m_{B-1}) , m_{p}\in\{0,\ldots,2^{k}-1\} where p\in\{0,\ldots,B-1\} and B=b/k denotes the total number of message symbols. 1) Recovering message positions. We first employ the same hash function and watermarking key used during message embedding stage to identify the message bit position p associated with each token y_{t} . All tokens assigned to the same index p are then grouped together for joint decoding of the corresponding 2^{k} -ary symbol m_{p} . 2) Computing confidence scores. For each message position p and each group j\in\{1,\ldots,k\} , we compute the empirical mean g -values over the associated token group, following Eq. 1 . Specifically, we obtain s^{p}_{j} and \bar{s}^{p}_{j} corresponding to \mathbf{g}_{j} and \mathbf{\bar{g}}_{j} , respectively. These quantities serve as confidence scores indicating whether the j -th bit of the 2^{k} -ary message symbol at position p favors 0 or 1 . 3) Extracting message symbols. For each message position p , we independently infer the j -th bit of the embedded symbol by comparing s^{p}_{j} and \bar{s}^{p}_{j} . The recovered 2^{k} -ray message symbol \hat{m}_{p}\in\{0,\ldots,2^{k}-1\} is then obtained as \hat{m}_{p}=\sum_{j=1}^{k}2^{k-j}\,\mathbb{I}\!\left(s^{p}_{j}<\bar{s}^{p}_{j}\right). (7) Computing the watermark z -score. We formulate watermark detection as a hypothesis test under the null hypothesis that the text is unwatermarked. Detection is based on a standardized z -score, whose definition and derivation are provided in Appendix G.4 . Under the null hypothesis, unwatermarked texts yield low z -score, whereas watermarked texts produce systematically larger values. Consequently, a sufficiently large z -score leads to rejection of the null hypothesis, indicating the presence of a watermark. Figure 3 : The comparison of multi-bit message decoding. Table 1 : Comparison of different multi-bit watermark performance, where denotes the baselines, denotes our watermark method. Bit Length Watermark Llama3-8B-Base Gemma2-9B-Base max 128 tokens max 256 tokens max 128 tokens max 256 tokens AUC \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow 16 bits MPAC 0.999 0.960 16.25 0.049 0.996 0.980 13.56 0.087 0.980 0.920 13.69 0.045 0.985 0.940 12.00 0.070 SegMark 0.993 0.948 15.88 0.551 0.995 0.999 12.94 0.839 0.979 0.895 13.19 1.234 0.999 0.977 10.94 2.402 BiMark 1.000 0.977 14.34 0.026 1.000 0.987 11.25 0.044 1.000 0.929 11.06 0.031 0.999 0.955 9.625 0.039 Ours ( k=1 ) 1.000 0.975 8.000 0.008 1.000 0.986 6.703 0.012 0.993 0.909 6.750 0.008 1.000 0.951 5.719 0.011 Ours ( k=2 ) 0.999 0.982 12.94 0.009 1.000 0.990 11.25 0.015 0.998 0.931 10.84 0.008 0.998 0.956 8.906 0.012 24 bits MPAC 0.996 0.916 16.75 0.052 0.997 0.955 14.00 0.074 0.972 0.875 13.69 0.044 0.959 0.905 12.00 0.070 SegMark 0.947 0.839 17.13 0.721 0.992 0.992 13.38 0.804 0.849 0.734 13.19 1.670 0.974 0.922 10.13 2.853 BiMark 1.000 0.941 14.34 0.027 1.000 0.970 11.34 0.036 0.998 0.875 11.63 0.030 0.990 0.921 9.938 0.043 Ours ( k=1 ) 1.000 0.931 8.125 0.008 1.000 0.963 6.625 0.014 0.998 0.880 7.078 0.008 0.998 0.916 5.844 0.013 Ours ( k=2 ) 1.000 0.943 13.69 0.009 1.000 0.972 11.53 0.014 0.998 0.894 10.44 0.008 0.999 0.925 8.625 0.012 32 bits MPAC 0.997 0.890 16.75 0.051 0.996 0.939 14.00 0.090 0.947 0.837 13.81 0.046 0.940 0.869 11.63 0.074 SegMark 0.909 0.802 16.63 0.642 0.989 0.964 12.94 1.053 0.824 0.691 12.75 1.751 0.955 0.868 10.25 3.362 BiMark 1.000 0.890 13.91 0.026 1.000 0.947 12.38 0.040 0.996 0.806 12.00 0.031 0.989 0.865 9.938 0.041 Ours ( k=1 ) 0.999 0.893 7.813 0.009 1.000 0.940 6.688 0.013 0.990 0.842 7.094 0.008 0.995 0.885 5.906 0.013 Ours ( k=2 ) 1.000 0.915 13.38 0.010 1.000 0.958 11.25 0.017 0.995 0.859 10.94 0.009 0.994 0.905 8.688 0.013 48 bits MPAC 0.993 0.828 16.63 0.057 0.990 0.891 13.19 0.078 0.936 0.770 14.00 0.048 0.914 0.821 12.19 0.076 SegMark 0.837 0.628 16.38 0.725 0.968 0.848 13.56 1.381 0.770 0.600 13.38 2.668 0.900 0.697 9.938 3.382 BiMark 1.000 0.783 14.25 0.027 0.999 0.880 11.63 0.037 0.985 0.688 11.53 0.029 0.975 0.767 10.03 0.040 Ours ( k=1 ) 0.997 0.838 7.938 0.011 1.000 0.897 6.594 0.014 0.987 0.768 6.875 0.009 0.994 0.822 5.656 0.014 Ours ( k=2 ) 1.000 0.862 13.69 0.011 1.000 0.916 11.63 0.017 0.993 0.800 10.34 0.010 0.989 0.851 9.063 0.015 4 Experiments Overview. We conduct a comprehensive empirical evaluation of WorldCup along three fundamental dimensions: message embedding capacity (Section 4.2 ), text quality (Section 4.4 ), and robustness (Section 4.4 ). 4.1 Experimental Setup Baselines. We compare WorldCup with representative multi-bit LLM watermarking methods that are publicly available, model-agnostic, and training-free, including BiMark ( 17 ) , MPAC ( 58 ) , and SegMark ( 46 ) . Message lengths are set to 16, 24, 32, and 48 bits. Detailed settings are in Appendix B . Datasets. Following prior work ( 30 ; 62 ) , we randomly sample 200 prompts from the C4 ( 47 ) and OpenGen ( 32 ) datasets for text generation. We further evaluate downstream performance on four representative tasks: machine translation, text summarization, question answering, and math reasoning. More details are in Appendix C . Metrics. We primarily use Bit Accuracy to evaluate message decoding correctness. Detectability is measured using AUC and Best F1 score, while text quality is assessed with Median Perplexity (PPL), ROUGE-L, BLEU, Pass@1, and the GPT-4 Score. Robustness is evaluated using AUROC curves. Metric definitions are provided in Appendix D . Models. For detectability and robustness evaluation, we use LLaMA3-8B-Base ( 21 ) and Gemma2-9B-Base ( 51 ) . For downstream tasks, we additionally evaluate their Instruct-tuned variants and the latest Ministral-8B-Instruct ( 39 ) . For fairness, we compute PPL using the larger Vicuna-13B-v1.5 model ( 9 ) . The details are in Appendix E . Implementation Details. Our method is implemented in Python 3.10.0 with PyTorch 2.6.0. All experiments are conducted on a single NVIDIA A100 80 GB GPU. We use a default setup with temperature = 1.0, top-k = 50, top-p = 0.95, no_repeat_ngram_size = 4, layers number m=30 , and \alpha=1.2 , together with lefthash and window size c=2 . Table 2 : The performance of various watermarking algorithms across four different downstream tasks on three instruct-tuned LLMs. Model Task 1: Short Q, Short A Task 2: Long Q, Short A Task 3: Short Q, Long A Task 4: Long Q, Long A Machine Translation Text Summarization Long-form QA Math Reasoning + Watermark Best F1 \uparrow Bit Accuracy \uparrow BLEU \uparrow Best F1 \uparrow Bit Accuracy \uparrow ROUGE-1 \uparrow Best F1 \uparrow Bit Accuracy \uparrow GPT4 Score \uparrow Best F1 \uparrow Bit Accuracy \uparrow Pass@1 \uparrow Ministral-8B-IT - - 0.200 ± 0.002 - - 0.221 ± 0.001 - - 5.250 ± 0.073 - - 0.340 ± 0.010 + MPAC 0.745 ± 0.005 0.886 ± 0.004 0.166 ± 0.001 0.951 ± 0.003 0.929 ± 0.002 0.215 ± 0.001 0.946 ± 0.002 0.879 ± 0.001 5.122 ± 0.029 0.785 ± 0.002 0.879 ± 0.001 0.265 ± 0.000 + BiMark 0.776 ± 0.006 0.803 ± 0.030 0.193 ± 0.003 0.977 ± 0.005 0.943 ± 0.005 0.208 ± 0.002 0.992 ± 0.000 0.894 ± 0.002 4.998 ± 0.088 0.767 ± 0.006 0.877 ± 0.002 0.310 ± 0.050 + WorldCup ( k=1 ) 0.672 ± 0.000 0.793 ± 0.000 0.221 ± 0.000 0.967 ± 0.008 0.948 ± 0.002 0.231 ± 0.002 0.977 ± 0.003 0.871 ± 0.003 5.319 ± 0.008 0.777 ± 0.002 0.854 ± 0.002 0.355 ± 0.000 + WorldCup ( k=2 ) 0.752 ± 0.004 0.919 ± 0.009 0.202 ± 0.005 0.983 ± 0.001 0.955 ± 0.002 0.213 ± 0.004 0.996 ± 0.001 0.905 ± 0.002 5.207 ± 0.092 0.898 ± 0.000 0.953 ± 0.000 0.315 ± 0.000 LLaMA3.1-8B-it - - 0.268 ± 0.004 - - 0.249 ± 0.001 - - 5.243 ± 0.069 - - 0.500 ± 0.020 + MPAC 0.700 ± 0.000 0.805 ± 0.002 0.224 ± 0.002 0.880 ± 0.010 0.869 ± 0.004 0.249 ± 0.002 0.940 ± 0.003 0.855 ± 0.003 5.110 ± 0.052 0.856 ± 0.006 0.927 ± 0.004 0.343 ± 0.007 + BiMark 0.718 ± 0.002 0.768 ± 0.068 0.258 ± 0.005 0.960 ± 0.004 0.898 ± 0.004 0.242 ± 0.001 0.981 ± 0.000 0.872 ± 0.005 5.066 ± 0.110 0.868 ± 0.005 0.922 ± 0.006 0.420 ± 0.020 + WorldCup ( k=1 ) 0.672 ± 0.000 0.768 ± 0.000 0.315 ± 0.000 0.912 ± 0.006 0.860 ± 0.004 0.261 ± 0.003 0.952 ± 0.006 0.843 ± 0.004 5.258 ± 0.062 0.814 ± 0.009 0.917 ± 0.004 0.468 ± 0.012 + WorldCup ( k=2 ) 0.705 ± 0.003 0.850 ± 0.025 0.272 ± 0.001 0.963 ± 0.004 0.919 ± 0.006 0.247 ± 0.002 0.986 ± 0.000 0.882 ± 0.005 5.149 ± 0.024 0.892 ± 0.003 0.947 ± 0.001 0.445 ± 0.005 Gemma2-9B-it - - 0.407 ± 0.001 - - 0.312 ± 0.002 - - 6.027 ± 0.093 - - 0.650 ± 0.010 + MPAC 0.673 ± 0.000 0.686 ± 0.007 0.393 ± 0.002 0.671 ± 0.000 0.619 ± 0.001 0.309 ± 0.001 0.838 ± 0.000 0.770 ± 0.001 5.920 ± 0.003 0.670 ± 0.000 0.670 ± 0.000 0.625 ± 0.000 + BiMark 0.675 ± 0.002 0.606 ± 0.027 0.398 ± 0.002 0.669 ± 0.000 0.598 ± 0.011 0.317 ± 0.003 0.883 ± 0.004 0.762 ± 0.001 5.946 ± 0.027 0.676 ± 0.002 0.706 ± 0.003 0.590 ± 0.000 + WorldCup ( k=1 ) 0.668 ± 0.000 0.623 ± 0.000 0.417 ± 0.001 0.671 ± 0.000 0.627 ± 0.001 0.318 ± 0.001 0.831 ± 0.004 0.760 ± 0.001 6.182 ± 0.003 0.676 ± 0.002 0.659 ± 0.005 0.630 ± 0.010 + WorldCup ( k=2 ) 0.673 ± 0.002 0.713 ± 0.002 0.361 ± 0.003 0.668 ± 0.001 0.664 ± 0.001 0.312 ± 0.001 0.886 ± 0.001 0.800 ± 0.002 6.136 ± 0.017 0.678 ± 0.005 0.755 ± 0.010 0.600 ± 0.020 4.2 Message Embedding Capacity Overall, we evaluate watermark detectability with different numbers of g -value functions, using k=1 and k=2 to embed 1-bit and 2-bit per token, respectively. Across all settings, WorldCup delivers both reliable watermark detection and accurate bit recovery, even as payload scales up. Multi-bit watermarking scenario. We evaluate message decoding performance under maximum sequence lengths of 128 and 256 tokens, with the minimum length fixed at 64. Table 1 shows that WorldCup consistently outperforms all baselines across both backbones. For LLaMA3-8B with 128-token sequences, WorldCup ( k=1 ) improves bit accuracy by 2.7% , 9.2% , 9.1% , and 21% for 16-bit, 24-bit, 32-bit, and 48-bit messages, respectively, compared to SegMark. For Gemma2-9B, WorldCup ( k=2 ) achieves average gains of 2.3% and 3.9% over MPAC and BiMark across all settings. Overall, these results indicate the effectiveness of WorldCup, where incorporating multiple g -functions improves per-token information utilization and leads to more reliable message recovery. We further observe that BiMark, MPAC, SegMark, WorldCup ( k=1 ), and WorldCup ( k=2 ) exhibit lower average decoding accuracy on Gemma2-9B than LLaMA3-8B, with decreases of 7.1% , 5.3% , 8.0% , 5.6% , and 5.2% , respectively. This gap is likely due to stronger generation capability, which reduces token-level uncertainly and details are in Appendix I.1 . Zero-bit watermarking scenario. Using the method described in Section 3.4 to compute the z -score as the detection statistic, WorldCup achieves an average AUC of approximately 99.7% across all scenarios. Moreover, 256-token sequences consistently outperform 128-token sequences, highlighting that longer contexts naturally accumulate stronger statistical evidence. These results not only confirm that detectability remains stable under large payloads, but also demonstrate that WorldCup gracefully subsumes the zero-bit watermarking regime without additional mechanisms. 4.3 Text Quality Preservation Perlexity (PPL). As shown in Table 1 and Fig. 7 , WorldCup consistently achieves lower perplexity than existing baselines, indicating strong text fluency preservation. We also observe that generated sequences with increased length tend to yield lower PPL. For instance, compared to 128-token sequences, longer 256-token sequences generated by WorldCup ( k=2 ) achieve average PPL reductions of 2.01 and 1.82 in LLaMA3-8B and Gemma2-9B, respectively. Notably, WorldCup ( k=1 ) even attains a PPL lower than that of natural text, suggesting that it implicitly regularizes generation and effectively mitigates distributional shifts from original LLM. Additional hyperparameter effects on PPL, such as no_repeat_ngram_size , are reported in Appendix I.6 . Downstream Task Performance. As summarized in Table 2 , WorldCup ( k=1 ) achieves the strongest overall performance across nearly all tasks. In particular, it improves Machine Translation and Text Summarization by an average of 2.6% and 0.93% , respectively, compared with non-watermarked baselines. This advantage stems from its multi-round tournament sampling mechanism, which robustly preserves the quality of generated text. Meanwhile, WorldCup ( k=2 ) attains the highest multi-bit decoding accuracy and outperforms previous baselines on several tasks, especially those involving longer outputs. For example, in the long-form QA task, it surpasses MPAC by an average of 2.6% , 2.7% , and 3.0% on Ministral, LLaMA3.1, and Gemma2, respectively. At the same time, on the LLaMA3.1-8B-IT model, WorldCup ( k=2 ) outperforms BiMark by 1.4% , 0.5% , 8.3% , and 2.5% across four downstream task metrics. These results demonstrate that our method not only achieves high decoding reliability but also maintains superior text quality, allowing flexible trade-offs to select the optimal strategy for specific downstream tasks. 4.4 Robustness to Real-world Attacks Attack Settings. We assess robustness under three common attacks ( 31 ) : editing (word deletion and synonym substitution), copy–paste, and paraphrasing using Dipper ( 32 ) using 256-token watermarked texts. Full settings are provided in Appendix F . As shown in Fig. 4 , Fig. 8 , and Fig. 9 , WorldCup consistently outperforms existing watermarking methods across a variety of attack scenarios, achieving higher AUC scores and decoding accuracy. In the six attack scenarios illustrated in Fig. 4 , WorldCup ( k=2 ) attains an average AUC of 98.9% and a decoding accuracy of 91.1% , surpassing BiMark, MPAC, and SegMark by 3.2% , 6.7% , and 15.1% , respectively. This robustness arises from its ability to more effectively leverage per-token redundancy to encode multiple bits, ensuring that a sufficient watermark signal remains even under substantial perturbations. Intuitively, as the number of embedded message bits increases, longer text sequences are required to achieve reliable message extraction and watermark detection; otherwise, fewer tokens per bit may lead to reduced robustness. We further investigate the effect of the hash window size in Appendix I.6 , demonstrating that increasing the window size diminishes robustness due to reduced locality in watermark signal aggregation. Figure 4 : The AUROC curves under different attacks on LLaMA3. Figure 5 : The ablation study of WorldCup ( k=2 ) on LLaMA3. 5 Further Analysis 5.1 Ablation Study Settings. To understand the contribution of each design choice in WorldCup ( k=2 ), i.e., q-\lambda\bar{q} , where q denotes the distribution induced by k groups of g -value functions, we perform a component-wise ablation study. We evaluate three variants: (i) removing the entropy-aware dynamic adjustment factor \lambda ( WorldCup w/o entropy , i.e., q-\bar{q} ); (ii) disabling the subtraction of complementary g -value distributions ( WorldCup w/o minus , i.e., q ); and (iii) replacing complementary g -value functions with randomly sampled ones ( WorldCup w/o complementary , i.e., q-\lambda q^{\prime} ). As shown in Fig. 5 , the entropy-aware factor plays a crucial role in preserving text quality. Although it slightly reduces bit decoding accuracy, it substantially lowers the PPL of the generated text. In contrast, directly sampling from the original g -values without subtracting their complementary counterparts achieves relatively low PPL, but suffers from poor discriminability, which significantly degrades decoding accuracy. Finally, when embedding either bit 0 or bit 1, using complementary g -value functions consistently outperforms random g -value functions in terms of both decoding accuracy and text quality. This observation is fully aligned with our theoretical analysis in Appendix G.4 . Overall, these results validate the necessity of each component and highlight their synergistic effect. 5.2 Computational Cost Analysis For encoding , we evaluate the encoding time and bit accuracy when embedding k bits per token using k groups of g -value functions, as shown in Fig. 6 . We observe that increasing k leads to a roughly linear increase in encoding time. Specifically, generating a single token takes 0.08 s when k=1 , while the time increases to 0.69 s when k=8 . Meanwhile, increasing k does not monotonically improve decoding accuracy, as the information capacity of each token is inherently limited. In practice, the best performance is typically achieved at k=2 . For decoding , we quantify the per sample decoding time, defined as the time required to decode one complete generated text, including both z -score computation and multi-bit message recovery. The results are reported in Table 1 . WorldCup achieves the highest decoding efficiency, requiring approximately \mathbf{1\sim 2~\mathrm{ms}} on average to decode a single watermarked text. This efficiency advantage arises from confidence-aware decoding, which enables fully parallel aggregation of g -values and removes the need for token counting. As a result, WorldCup scales favorably to longer sequences and higher payloads, highlighting its suitability for practical large-scale deployment. 5.3 Different Watermark Functions and Detectors We further examine how different g -value functions and detectors affect multi-bit watermark performance. Non-distortionary g -value functions preserve the sampling distribution in expectation and thus improve text quality, but they substantially reduce watermark detectability and multi-bit decoding accuracy, as shown in Appendix I.2 . We also observe that weighted-mean detectors perform better when k=1 , whereas mean detectors are more effective for k=2 , as detailed in Appendix I.3 . This may be because combining multiple g -value functions smooths layer-wise effects and mitigates sensitivity to tournament depth, with supporting experiments on varying sampling layers m in Appendix I.4 . Figure 6 : The effect of embedding k bits in each token. 6 Conclusion We introduced WorldCup, a novel multi-bit watermarking framework for LLMs that treats inference-time sampling as a communication channel for structured information embedding. By integrating complementary watermarking signals with entropy-aware modulation and confidence-aware decoding, WorldCup enables robust and scalable message embedding without compromising text quality. Experiments across diverse models and tasks demonstrate clear advantages over existing baselines in accuracy, efficiency, and robustness. We believe that WorldCup provides a principled foundation for scalable multi-bit watermarking and supports practical deployment for reliable LLM content attribution. Impact Statement This paper presents WorldCup, a multi-bit watermarking framework for large language models that enables reliable content attribution and integrity verification. By supporting robust message embedding while preserving text quality, WorldCup promotes accountable and transparent LLM deployment. At the same time, we acknowledge potential risks such as privacy concerns or misuse for surveillance and encourage responsible research and governance to ensure watermarking serves transparency and user rights. Appendix A Related Work Existing LLM watermarking methods can be broadly categorized into zero-bit watermarking and multi-bit watermarking approaches ( 38 ) , depending on whether message bits are explicitly embedded. A.1 Zero-bit Watermarking The pioneering KGW method ( 30 ) first introduced token-level watermarking by partitioning the vocabulary into ”green” and ”red” token lists and modifying the logits distribution during inference to embed a watermark signal. To enchance robustness, Unigram ( 62 ) adopted a globally fixed vocabulary partition, while subsequent works ( 37 ; 49 ; 26 ; 25 ) leveraged semantic and frequency-based features to defend against editing attacks. To mitigate text quality degradation, several studies proposed entropy-based watermarking schemes ( 40 ; 34 ; 55 ) , while others explored unbiased reweighting strategies ( 56 ; 27 ; 8 ) . Additionally, 13 ; 1 ; 19 designed alternative token sampling mechanisms that preserve the the original logits distribution. Overall, zero-bit watermarking can only determine whether a text contains a watermark, limited in broader scenarios such as content provenance tracing. A.2 Multi-bit Watermarking 18 ; 28 embedded multi-bit message by establishing a mapping between predefined watermark keys and message bits. 54 ; 11 divides both the text and message into multiple independent blocks, sequentially encoding each message segment into a corresponding text block. Although these methods achieve moderate decoding accuracy, they require enumerating all candidate messages during decoding, resulting in high exponential computational complexity with respect to message length (i.e., O(2^{b}) for b bits). To overcome this limitation, 61 ; 57 ; 33 explore training-based and post-hoc approaches, while 59 propose a black-box method for generating sentence-level watermarks. In contrast, MPAC ( 58 ) assigns distinct message bits to different tokens via hash functions and extends the KGW framework accordingly. Building upon this idea, BiMark ( 17 ) and StealthInk ( 29 ) , 60 ; 5 developed distortion-free multi-bit watermark variants, while 46 introduced error-correcting codes ( 6 ; 15 ; 35 ) to further enhance decoding robustness. Despite these advances, current multi-bit watermarking techniques still struggle to jointly optimize capacity, decoding accuracy, efficiency, and text quality. Table 3 : The comparison between WorldCup (our method) and related studies, categorized as yes ( ✔ ), partial ( P ), or no ( ✗ ). Multi-bit Watermarking Detectability Text Quality Robustness Token Capacity Accuracy Efficiency MPAC ( 58 ) ✔ P ✗ ✔ ✔ ✔ BiMark ( 17 ) ✔ ✔ P P ✔ ✔ SegMark ( 46 ) ✔ P ✔ ✔ ✔ ✗ StealthInk ( 29 ) ✔ ✔ ✗ ✔ P ✔ WorldCup (Ours) ✔ ✔ ✔ ✔ ✔ ✔ Appendix B Baselines For each baseline, we follow the configurations in original papers. The key hyperparameter settings are as follows: • BiMark ( 17 ) : The base scaling factor \tilde{\delta} is set to 1.0 , and the number of layers d is 10 . The proportion \gamma of green lists is 0.5 and the window size is 2. The values of \text{c}\_\text{key} and \text{bit}\_\text{idx}\_\text{key} are 530773 and 283519, respectively. • MPAC ( 58 ) : We adopt the lefthash scheme. The window size is 2, and the proportion \gamma of green lists is 0.5. A bias \delta=2.0 is added to the logit scores of green tokens. The hash key is 15485863. • SegMark ( 46 ) : We use the RSBH scheme (balanced segment assignment with ECC). The window size is 2, the proportion \gamma of green lists is 0.5, and a bias \delta=2.0 is added to the logit of green tokens. The salt key is 35317. Appendix C Datasets We follow previous work ( 52 ) to evaluate our multi-bit watermark method on the following datasets: • C4 ( 47 ) dataset is a large-scale, high-quality English pretraining corpus constructed by Google from Common Crawl. After extensive cleaning and filtering to remove non-linguistic, low-quality, and duplicate content, it yields roughly 750 GB of clean English text. We use the processed version in https://huggingface.co/datasets/allenai/c4 . • OpenGen ( 32 ) dataset contains 3,000 two-sentence text blocks drawn from the validation split of WikiText-103 ( 42 ) , with the subsequent 300 tokens written by human. We sample 200 instances from this dataset for our experiments. The dataset are in https://github.com/XuandongZhao/Unigram-Watermark . • WMT ( 3 ) dataset is a widely recognized benchmark in machine translation, containing parallel corpora from diverse sources and covering multiple language pairs. For our downstream evaluation, we primarily collect 200 samples from the WMT’19 De-En subset, with decoding parameters set to max_new_tokens = 64 and min_new_tokens = 16 ( short input, short output ). We embed 2-bit message into each generated sample. It can be found at the following link: https://huggingface.co/datasets/wmt/wmt19/viewer/de-en/validation . • CNN_DailyMail ( 24 ) dataset is a large-scale English news corpus containing over 300,000 unique articles written by journalists from CNN and the Daily Mail. The current release supports both extractive and abstractive summarization. We collect 200 samples and prompt the model to produce a one-sentence summary, with max_new_tokens = 64 and min_new_tokens = 32 ( long input, short output ), and embed a 16-bit message into each generated summary. Details of the dataset can be found at: https://huggingface.co/datasets/abisee/cnn_dailymail . • ELI5 ( 16 ) dataset is a long-form QA dataset sourced from the Reddit community “Explain Like I’m Five.” It contains 270k diverse questions that require multi-sentence, explanatory answers supported by web evidence. We get 200 data points and set max_new_tokens = 256 and min_new_tokens = 64 ( short input, long output ), and embed a 32-bit message into each generated answer. We use the processed subset available at https://github.com/THU-KEG/WaterBench/blob/main/data/WaterBench/2-1_longform_qa.jsonl . • GSM8K ( 10 ) dataset consists of 8,500 high-quality grade-school math word problems requiring 2–8 steps of reasoning, with answers expressed in natural language. For downstream evaluation, we sample 200 instances and use an 8-shot setting with max_new_tokens = 256 and min_new_tokens = 64 ( long input, long output ), and embed a 4-bit message into each generated solution. The dataset is available at: https://huggingface.co/datasets/openai/gsm8k . Appendix D Metrics All evaluation metrics used in our experiments are described in detail below: • Bit Accuracy (Bit Acc.) measures the proportion of correctly extracted bits across all samples. Let \mathbf{m}^{(j)}=(m^{(j)}_{1},\dots,m^{(j)}_{L}) denote the embedded message of length L for the j -th sample, and \hat{\mathbf{m}}^{(j)}=(\hat{m}^{(j)}_{1},\dots,\hat{m}^{(j)}_{L}) be the corresponding extracted message. The Bit Accuracy over n samples is defined as \mathrm{Bit\ Acc.}=\frac{1}{nL}\sum_{j=1}^{n}\sum_{i=1}^{L}\mathbb{I}\big[\hat{m}^{(j)}_{i}=m^{(j)}_{i}\big] (8) where \mathbb{I}[\cdot] denotes the indicator function. It reflects fine-grained bit-level decoding performance, as shown in Table 1 . • Message Extracted Rate (ME Rate) quantifies the probability of perfectly recovering the entire embedded message. A message is considered successfully extracted if and only if all its bits are correctly recovered. Formally, the Message Extracted Rate over n samples is defined as: \mathrm{ME\ Rate}=\frac{1}{n}\sum_{j=1}^{n}\mathbb{I}\big[\hat{\mathbf{m}}^{(j)}=\mathbf{m}^{(j)}\big]. (9) It is a strict metric that penalizes any bit error and reflects end-to-end message recovery reliability, as shown in Table 4 . • F1 Score is the harmonic mean of precision and recall: \mathrm{F1}=2\cdot\frac{\mathrm{Precision}\cdot\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}} (10) Best F1 Score denotes the maximum F1 score obtained over a threshold sweep, commonly used for evaluating binary classifiers without fixing a specific decision threshold. • AUROC Curve (Receiver Operating Characteristic) plots TPR (True Positive Rate) against FPR (False Positive Rate) under varying decision thresholds. The area under this curve ( AUC ) summarizes the watermark detector’s ranking ability, with values closer to 1 indicating stronger discriminative performance. • Perplexity (PPL) measures how well a language model predicts predicts a given text. For a sequence x_{1:T} : \mathrm{PPL}=\exp\!\left(-\frac{1}{T}\sum_{t=1}^{T}\log p(x_{t}\mid x_{0 in the following derivation. In a single-layer tournament, N samples participate in the selection. Let |Y^{\ast}|=j denote the number of tokens in the winning set, and suppose that x_{t} appears i times among these j tokens (pairwise comparison is not required). The probability that outputs x_{t} is: \displaystyle\mathbb{P}\!\left(\text{Alg.~\ref{Algorthim: Binary WorldCup Watermarking} returns }x_{t}\right) \displaystyle=\sum_{j=1}^{N}\sum_{i=1}^{j}\mathbb{P}\!\left(|Y^{\ast}|=j,\,x_{t}\text{ appears }i\text{ times in }Y^{\ast},\,\text{Alg.~\ref{Algorthim: Binary WorldCup Watermarking} returns }x_{t}\right) (22) \displaystyle=\sum_{j=1}^{N}\sum_{i=1}^{j}\binom{N}{j}p\!\left(V^{2 bits. Let \mathbf{m}^{\prime}\in\{0,1\}^{k} denote a k -bit message, and let \{q_{0},q_{1},\ldots,q_{k-1}\} be the corresponding g -value functions \mathbf{g}_{i} or probability subsets associated with each bit. Then, the unnormalized watermark distribution for a message \mathbf{m}^{\prime} can be written as: P_{\Theta,\mathbf{m}}=\sum_{i=0}^{k-1}\big(b_{i}\overline{q}_{i}+(1-b_{i})q_{i}\big)-\lambda\sum_{i=0}^{k-1}\big(b_{i}q_{i}+(1-b_{i})\overline{q}_{i}\big) (31) where b_{i} is the i -th bit of \mathbf{m}^{\prime} , and \overline{q_{i}} is the distribution induced by the complementary g -value function \bar{\mathbf{g}}_{i} . G.4 Compute Watermark Z-score Let \mathbf{y}=(y_{1},\ldots,y_{T}) denote a generated text of length T produced by an m -layer WorldCup sampling scheme. As in multi-bit watermarking, each token y_{t} is deterministically mapped, via the shared hash function and watermarking key, to a message bit position p_{t}\in\{1,\ldots,b\} . For each token y_{t} and layer \ell\in\{1,\ldots,m\} , we evaluate complementary g -value functions g_{\ell}(y_{t},r)\in\{0,1\} and \bar{g}_{\ell}(y_{t},r)=1-g_{\ell}(y_{t},r) , and define the signed token–layer score: d_{t,\ell}\triangleq g_{\ell}(y_{t},r)-\bar{g}_{\ell}(y_{t},r)=2g_{\ell}(y_{t},r)-1 (32) Rather than aggregating all tokens simultaneously, we compute the detection statistic in a bit-wise, cumulative manner. Let \mathcal{T}_{p}=\{t:p_{t}=p\} denote the set of tokens assigned to bit position p , and define the cumulative token set up to position p as \mathcal{C}_{p}=\bigcup_{i=1}^{p}\mathcal{T}_{i} . The confidence-aware statistic after incorporating the first p bit positions is defined as S_{p}(\mathbf{y})=\frac{1}{m|\mathcal{C}_{p}|}\sum_{t\in\mathcal{C}_{p}}\sum_{\ell=1}^{m}d_{t,\ell} (33) As p increases, token–layer contributions are progressively accumulated. Under the null hypothesis H_{0} of unwatermarked text, the complementary g -value functions are symmetric and exchangeable, yielding \mathbb{E}_{H_{0}}[g_{\ell}(y_{t},r)]=\mathbb{E}_{H_{0}}[\bar{g}_{\ell}(y_{t},r)]=0.5 . Consequently, \mathbb{E}_{H_{0}}[d_{t,\ell}]=0 and thus \mathbb{E}_{H_{0}}[S_{p}(\mathbf{y})]=0 for all p . Moreover, since d_{t,\ell}\in\{-1,+1\} with equal probability under H_{0} , we have \operatorname{Var}_{H_{0}}(d_{t,\ell})=1 . Assuming approximate independence across token–layer pairs (t,\ell) , the variance of the cumulative statistic is \operatorname{Var}_{H_{0}}(S_{p}(\mathbf{y}))=\frac{1}{m|\mathcal{C}_{p}|} . We therefore define a position-dependent standardized detection statistic z_{p}=\frac{S_{p}(\mathbf{y})}{\sqrt{1/(m|\mathcal{C}_{p}|)}}=\sqrt{m|\mathcal{C}_{p}|}~S_{p}(\mathbf{y}), (34) which measures the watermark strength after cumulatively incorporating all tokens mapped to the first p bit positions. In particular, z_{b} corresponds to the final z -score computed over the entire text. By the central limit theorem, z_{p} converges in distribution to \mathcal{N}(0,1) under the null hypothesis H_{0} for each p. For watermarked text, tokens aligned with the embedded message bits contribute biased scores within each bit position, causing the cumulative statistic to deviate from zero as additional positions are incorporated. In multi-bit watermarking with symmetric signed scores d_{t,\ell}\in\{-1,+1\} , the overall mean bias may cancel out when the numbers of embedded 0- and 1-bits are balanced, making the final signed statistic z_{b} close to zero. However, the watermark does not induce a uniform mean shift; instead, it creates structured, position-dependent biases. Consequently, detection remains effective when using two-sided or position-aware statistics, such as |z_{p}| , \max_{p}|z_{p}| , or the energy \sum_{p=1}^{b}z_{p}^{2} , which are robust to sign cancellation. In practice, we find that replacing d_{t,\ell} with a max-based heuristic, d_{t,\ell}\triangleq\max(g_{\ell}(y_{t},r),\bar{g}_{\ell}(y_{t},r)) , often yields stronger empirical detection performance. While this hard selection improves the effective signal-to-noise ratio by suppressing noisy contributions, it introduces a positive bias under H_{0} and therefore does not yield a properly calibrated z -score. Appendix H More Experimental Results Table 4 : Comparison of different multi-bit watermark performance, where denotes the baselines, denotes our watermark method. Bit Length Watermark Llama3-8B-Base Gemma2-9B-Base max 128 tokens max 256 tokens max 128 tokens max 256 tokens AUC \uparrow ME Rate \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow ME Rate \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow ME Rate \uparrow PPL \downarrow Time (s) \downarrow AUC \uparrow ME Rate \uparrow PPL \downarrow Time (s) \downarrow 16 bits MPAC 0.999 0.520 16.25 0.049 0.996 0.785 13.56 0.087 0.980 0.305 13.69 0.045 0.985 0.500 12.00 0.070 SegMark 0.993 0.815 15.88 0.551 0.995 0.995 12.94 0.839 0.979 0.510 13.19 1.234 0.999 0.905 10.94 2.402 BiMark 1.000 0.750 14.34 0.026 1.000 0.860 11.25 0.044 1.000 0.410 11.06 0.031 0.999 0.620 9.625 0.039 Ours ( k=1 ) 1.000 0.915 7.938 0.007 1.000 0.970 6.563 0.012 0.998 0.710 7.250 0.007 0.994 0.860 6.344 0.012 Ours ( k=2 ) 0.998 0.940 12.94 0.008 1.000 0.990 10.94 0.013 1.000 0.850 10.94 0.009 1.000 0.965 9.563 0.012 24 bits MPAC 0.996 0.175 16.75 0.052 0.997 0.445 14.00 0.074 0.972 0.085 13.69 0.044 0.959 0.190 12.00 0.070 SegMark 0.947 0.450 17.13 0.721 0.992 0.975 13.38 0.804 0.849 0.190 13.19 1.670 0.974 0.695 10.13 2.853 BiMark 1.000 0.280 14.34 0.027 1.000 0.700 11.34 0.036 0.998 0.085 11.63 0.030 0.990 0.415 9.938 0.043 Ours ( k=1 ) 0.999 0.505 7.938 0.008 0.996 0.820 6.625 0.016 0.998 0.280 7.375 0.008 0.993 0.610 5.844 0.013 Ours ( k=2 ) 1.000 0.710 12.84 0.008 1.000 0.885 10.75 0.013 0.999 0.500 10.44 0.008 1.000 0.815 9.125 0.016 32 bits MPAC 0.997 0.035 16.75 0.051 0.996 0.255 14.00 0.090 0.947 0.005 13.81 0.046 0.940 0.025 11.63 0.074 SegMark 0.909 0.240 16.63 0.642 0.989 0.840 12.94 1.053 0.824 0.025 12.75 1.751 0.955 0.465 10.25 3.362 BiMark 1.000 0.055 13.91 0.026 1.000 0.425 12.38 0.040 0.996 0.005 12.00 0.031 0.989 0.115 9.938 0.041 Ours ( k=1 ) 0.996 0.100 8.250 0.008 1.000 0.565 6.547 0.014 0.996 0.060 7.125 0.008 0.998 0.355 6.250 0.015 Ours ( k=2 ) 0.998 0.390 13.28 0.011 1.000 0.800 10.56 0.013 1.000 0.105 10.66 0.009 1.000 0.565 9.500 0.013 48 bits MPAC 0.993 0.000 16.63 0.057 0.990 0.010 13.19 0.078 0.936 0.000 14.00 0.048 0.914 0.000 12.19 0.076 SegMark 0.837 0.000 16.38 0.725 0.968 0.340 13.56 1.381 0.770 0.000 13.38 2.668 0.900 0.030 9.938 3.382 BiMark 1.000 0.000 14.25 0.027 0.999 0.060 11.63 0.037 0.985 0.000 11.53 0.029 0.975 0.000 10.03 0.040 Ours ( k=1 ) 0.998 0.000 7.750 0.009 1.000 0.140 6.688 0.014 0.983 0.000 7.000 0.009 0.985 0.010 5.938 0.013 Ours ( k=2 ) 0.999 0.025 12.84 0.009 1.000 0.310 11.44 0.014 0.996 0.015 10.50 0.009 0.998 0.130 9.313 0.014 To evaluate message-level decoding accuracy, we use the Message Extracted Rate (ME Rate). We further apply a Hamming error-correcting code ( 23 ) to alleviate the sensitivity of ME Rate to a small number of bit errors. As reported in Table 4 , WorldCup consistently outperforms existing baselines across most settings. Fig. 7 shows that our method achieves more reliable detection while preserving better text quality. Additional robustness results (Fig. 8 and Fig. 9 ) indicate that sentence-level attacks (e.g., back-translation and paraphrasing) are more harmful to multi-bit watermarking than word-level attacks (e.g., word deletion and synonym substitution). This is inherent to bit-allocation-based schemes, as sentence-level transformations disrupt the contextual alignment required to recover token-level bit information. Improving robustness to such attacks remains an open challenge. Figure 7 : The comparison of PPL across different multi-bit watermarking methods on LLaMA3-8B and Gemma2-9B. Figure 8 : The AUROC curves under different attacks on LLaMA3-8B-Base model (16 bits). Figure 9 : The AUROC curves under different attacks on Gemma2-9B-Base model (16 bits). The numerical results corresponding to Fig. 5 and Fig. 6 are reported in Table 5 and Table 6 , respectively. Based on these experimental results, we select the hyperparameter \alpha=1.2 and set the number of g -value functions to 2, as this configuration provides a favorable trade-off among bit accuracy, watermark detectability, and text quality. Table 5 : The numerical results of the ablation study on different components of WorldCup. Model C4 Dataset 16 bits 24 bits 32 bits 48 bits 64 bits + Watermark Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Llama3-8B-Base + WorldCup ( k=1 , ours) 0.997 0.986 6.688 0.997 0.958 6.688 0.992 0.930 6.844 0.998 0.880 6.563 0.997 0.807 6.594 + WorldCup ( k=1 , w/o comp.) 0.995 0.949 6.813 1.000 0.907 6.656 1.000 0.874 7.031 1.000 0.797 6.563 0.998 0.754 6.750 + WorldCup ( k=2 , w/o entropy) 1.000 0.996 28.38 1.000 0.989 28.75 1.000 0.983 28.75 1.000 0.963 27.00 0.998 0.929 28.75 + WorldCup ( k=2 , w/o minus) 0.940 0.933 5.906 0.946 0.895 6.094 0.910 0.853 6.063 0.884 0.802 6.063 0.870 0.765 5.875 + WorldCup ( k=2 , w/o comp.) 0.995 0.986 14.13 0.995 0.975 13.56 0.995 0.937 13.56 0.992 0.904 14.13 0.987 0.868 13.19 + WorldCup ( k=2 , ours, \lambda ) 0.995 0.988 9.813 0.980 0.974 10.13 0.985 0.956 9.313 0.953 0.923 9.813 0.954 0.873 9.406 + WorldCup ( k=2 , ours, 1.2\lambda ) 0.990 0.990 11.25 0.985 0.977 11.25 0.988 0.965 10.94 0.970 0.930 11.06 0.963 0.888 11.44 + WorldCup ( k=2 , ours, 1.5\lambda ) 0.998 0.995 14.81 0.992 0.981 14.81 0.993 0.973 15.38 0.985 0.936 15.38 0.975 0.909 14.44 + WorldCup ( k=2 , ours, 2.0\lambda ) 1.000 0.998 49.00 1.000 0.995 49.75 1.000 0.989 48.63 1.000 0.963 49.38 1.000 0.938 49.38 Table 6 : The effect of g -value function number on differnet LLMs. Model g _function_num=1 g _function_num=2 g _function_num=3 g _function_num=4 g _function_num=6 g _function_num=8 Bit Acc \uparrow Enc Time \downarrow Bit Acc \uparrow Enc Time \downarrow Bit Acc \uparrow Enc Time \downarrow Bit Acc \uparrow Enc Time \downarrow Bit Acc \uparrow Enc Time \downarrow Bit Acc \uparrow Enc Time \downarrow Llama3-8B-Base - - - - - - - - - - - - WorldCup (24 bit) 0.980 0.083 0.995 0.213 0.993 0.287 0.990 0.371 0.986 0.533 0.985 0.690 WorldCup (48 bit) 0.917 0.069 0.975 0.203 0.966 0.285 0.958 0.366 0.950 0.530 0.939 0.689 Gemma2-9B-Base - - - - - - - - - - - - WorldCup (24 bit) 0.943 0.079 0.991 0.214 0.988 0.296 0.986 0.374 0.975 0.532 0.971 0.691 WorldCup (48 bit) 0.871 0.080 0.951 0.215 0.945 0.294 0.933 0.377 0.926 0.531 0.910 0.693 Appendix I Additional Analysis I.1 Entropy Analysis From the results in Table 1 , we observe that existing multi-bit watermarking methods achieve substantially lower bit decoding accuracy on Gemma2-9B-Base than on LLaMA3-8B-Base. To better understand this phenomenon, we conduct an entropy-based analysis. Specifically, we visualize the average token-level entropy for each sample during text generation, as illustrated in Fig. 10 . The results clearly indicate that Gemma2-9B-Base produces tokens with consistently lower entropy, aligning with prior findings that low-entropy text is inherently more challenging for watermark embedding. A plausible explanation is that Gemma2-9B-Base, owing to its larger parameter scale compared to LLaMA3-8B-Base, generates tokens with higher confidence, thereby reducing entropy and limiting the effective embedding capacity for multi-bit watermarks. Figure 10 : The comparison of spike entropy between LLaMA3-8B-Base and Gemma2-9B-Base across different message bits I.2 Distortionary and Non-distortionary g -value Functions According to SynthID ( 13 ) , we define distortionary and non-distortionary g -value functions as follows: Definition I.1 . A sampling algorithm \mathcal{S}:\Delta\mathcal{V}\times\mathcal{R}\rightarrow\mathcal{V} is (single-token) non-distortionary if for any probability distribution p\in\Delta\mathcal{V} and token x\in\mathcal{V} : \mathbb{E}_{r\sim\text{Unif}{(\mathcal{R})}}[\mathbb{P}(\mathcal{S}(p,r)=x)]=p(x) (35) If \mathcal{S} is not non-distortionary, we call it distortionary. Therefore, following the design of non-distortionary and distortionary g -value functions in SynthID, we evaluate their performance in the multi-bit watermarking setting. The results are summarized in Fig. 11 and Table 7 . Specifically, WorldCup-v1, v2, v3, and v4 in the figure correspond to the configurations ( k=1 , non-distortionary), ( k=1 , distortionary), ( k=2 , non-distortionary), and ( k=2 , distortionary) in the Table 7 , respectively. We observe that the non-distortionary setting yields higher cosine similarity between the generated watermarked text and the ground truth compared to the distortionary setting, at the cost of reduced bit accuracy. Consequently, in practical scenarios, the choice between these two settings can be made based on application-specific requirements. Unless otherwise stated, our experiments primarily adopt the distortionary configuration. Table 7 : The comparison of distortionary g -value function and non-distortionary g -value function. Model C4 Dataset 50 tokens 100 tokens 150 tokens 200 tokens 250 tokens + Watermark Best F1 \uparrow Bit Acc \uparrow Cos Sim \uparrow Best F1 \uparrow Bit Acc \uparrow Cos Sim \uparrow Best F1 \uparrow Bit Acc \uparrow Cos Sim \uparrow Best F1 \uparrow Bit Acc \uparrow Cos Sim \uparrow Best F1 \uparrow Bit Acc \uparrow Cos Sim \uparrow Llama3-8B-Base + BiMark 0.972 0.513 0.422 0.998 0.743 0.457 1.000 0.851 0.487 1.000 0.911 0.510 1.000 0.943 0.515 + WorldCup ( k=1 , non-distortionary) 0.883 0.463 0.425 0.916 0.647 0.452 0.883 0.719 0.468 0.881 0.754 0.496 0.869 0.767 0.497 + WorldCup ( k=1 , distortionary) 0.935 0.494 0.425 0.967 0.705 0.449 0.980 0.799 0.471 0.992 0.858 0.490 0.990 0.889 0.488 + WorldCup ( k=2 , non-distortionary) 0.934 0.673 0.395 0.941 0.812 0.433 0.929 0.853 0.454 0.915 0.871 0.479 0.912 0.879 0.478 + WorldCup ( k=2 , distortionary) 0.947 0.689 0.397 0.995 0.853 0.426 0.997 0.914 0.451 1.000 0.949 0.478 1.000 0.968 0.481 Gemma2-9B-Base + BiMark 0.949 0.480 0.430 0.988 0.706 0.466 0.995 0.816 0.490 1.000 0.879 0.505 0.998 0.909 0.504 + WorldCup ( k=1 , non-distortionary) 0.807 0.439 0.439 0.864 0.620 0.493 0.888 0.699 0.514 0.909 0.741 0.532 0.879 0.760 0.536 + WorldCup ( k=1 , distortionary) 0.865 0.455 0.438 0.935 0.653 0.478 0.948 0.747 0.506 0.972 0.803 0.532 0.975 0.838 0.533 + WorldCup ( k=2 , non-distortionary) 0.870 0.630 0.415 0.950 0.780 0.454 0.958 0.834 0.479 0.962 0.861 0.504 0.959 0.876 0.508 + WorldCup ( k=2 , distortionary) 0.915 0.665 0.412 0.992 0.825 0.459 0.990 0.888 0.477 0.993 0.920 0.503 0.997 0.942 0.505 Figure 11 : The comparison of distortionary and non-distortionary g -value functions I.3 Different Detectors Following SynthID Text ( 13 ) , we consider two watermark detectors: mean detector (D1) and weighted-mean detector (D2). The mean detector computes the average g -value across all tokens and all layers as \text{MeanScore}(x):=\frac{1}{mT}\sum_{t=1}^{T}\sum_{\ell=1}^{m}g_{t,\ell}, (36) where T denotes the number of tokens and m the number of layers. The weighted-mean detector assigns non-increasing weights \alpha_{1}\geq\cdots\geq\alpha_{m}\geq 0 to different layers, with \sum_{\ell=1}^{m}\alpha_{\ell}=m : \text{WeightedMeanScore}(x,\boldsymbol{\alpha}):=\frac{1}{mT}\sum_{t=1}^{T}\sum_{\ell=1}^{m}\alpha_{\ell},g_{t,\ell}. (37) Specifically, we set \alpha_{1}=\tau,\quad\alpha_{2}=\tau-\frac{\tau-\mu}{m-1},\quad\alpha_{3}=\tau-2\frac{\tau-\mu}{m-1},\ \ldots,\ \alpha_{m}=\mu , with \tau=10 and \mu=1 , and then renormalize the weights such that \sum_{\ell=1}^{m}\alpha_{\ell}=m . We evaluate both detectors under k=1 and k=2 on the C4 and OpenGen datasets across varying token lengths. Figure 12 shows the corresponding detection curves, and the quantitative results are summarized in Table 8 . Our results indicate that for k=1 , the weighted-mean detector consistently outperforms the mean detector, in agreement with the findings of SynthID. This is because the contribution of watermarking evidence from each layer diminishes as depth increases, making layer-wise weighting beneficial. However, for k=2 , this trend reverses, and the mean detector achieves superior performance. A plausible explanation is that the use of multiple g-functions alleviates the attenuation of watermark signals across layers, thereby diminishing the advantage of layer weighting. Table 8 : The comparison of mean watermark detector and weighted mean watermark detector on WorldCup. Dataset Llama3-8B-Base 50 tokens 100 tokens 150 tokens 200 tokens 250 tokens + Watermark Best F1 \uparrow Bit Acc \uparrow Best F1 \uparrow Bit Acc \uparrow Best F1 \uparrow Bit Acc \uparrow Best F1 \uparrow Bit Acc \uparrow Best F1 \uparrow Bit Acc \uparrow C4 Dataset - - - - - - - - - - + BiMark 1.000 ± 0.000 0.530 ± 0.001 1.000 ± 0.000 0.758 ± 0.007 1.000 ± 0.000 0.866 ±0.007 1.000 ± 0.000 0.919 ± 0.006 1.000 ± 0.000 0.949 ± 0.008 + WorldCup ( k=1 , D1) 0.931 ± 0.039 0.509 ± 0.009 0.980 ± 0.010 0.736 ±0.010 0.985 ± 0.015 0.832 ± 0.002 0.995 ± 0.005 0.879 ± 0.002 0.995 ± 0.005 0.906 ± 0.001 + WorldCup ( k=1 , D2) 0.953 ± 0.005 0.520 ± 0.007 0.985 ± 0.005 0.744 ± 0.005 0.995 ± 0.005 0.846 ± 0.001 0.990 ± 0.010 0.895 ± 0.004 0.995 ± 0.005 0.918 ± 0.000 + WorldCup ( k=2 , D1) 0.985 ± 0.015 0.697 ± 0.004 0.995 ± 0.005 0.869 ± 0.001 1.000 ± 0.000 0.930 ± 0.002 1.000 ± 0.000 0.957 ± 0.003 1.000 ± 0.000 0.974 ± 0.001 + WorldCup ( k=2 , D2) 0.975 ± 0.015 0.683 ± 0.003 1.000 ± 0.000 0.857 ± 0.006 1.000 ± 0.000 0.914 ± 0.001 1.000 ± 0.000 0.948 ± 0.003 1.000 ± 0.000 0.966 ± 0.002 OpenGen Dataset + BiMark 0.995 ± 0.005 0.538 ± 0.008 1.000 ± 0.000 0.765 ± 0.004 1.000 ± 0.000 0.774 ± 0.006 1.000 ± 0.000 0.927 ± 0.006 1.000 ± 0.000 0.955 ± 0.003 + WorldCup ( k=1 , D1) 0.934 ± 0.006 0.517 ± 0.004 0.975 ± 0.015 0.749 ± 0.006 1.000 ± 0.000 0.848 ± 0.006 1.000 ± 0.000 0.896 ± 0.004 1.000 ± 0.000 0.927 ± 0.004 + WorldCup ( k=1 , D2) 0.960 ± 0.020 0.526 ± 0.011 0.995 ± 0.005 0.768 ± 0.006 1.000 ± 0.000 0.868 ± 0.001 1.000 ± 0.000 0.915 ± 0.002 1.000 ± 0.000 0.943 ± 0.000 + WorldCup ( k=2 , D1) 0.975 ± 0.005 0.702 ± 0.005 1.000 ± 0.000 0.871 ± 0.005 1.000 ± 0.000 0.930 ± 0.004 1.000 ± 0.000 0.959 ± 0.001 1.000 ± 0.000 0.972 ± 0.006 + WorldCup ( k=2 , D2) 0.985 ± 0.005 0.684 ± 0.004 1.000 ± 0.000 0.850 ± 0.003 1.000 ± 0.000 0.909 ± 0.002 1.000 ± 0.000 0.949 ± 0.003 1.000 ± 0.000 0.966 ± 0.000 Figure 12 : The comparison of mean detector and weighted mean detector. I.4 WorldCup Sampling Layers To investigate the impact of the number of layers in WorldCup, we conducted experiments with different layer counts m=5,10,20,30,50 on LLaMA3-8B-Base and Gemma2-9B-Base, with results summarized in Table 9 . Empirically, a layer count of m=30 yields relatively optimal performance, consistent with findings from SynthID-Text ( 13 ) . Lower layer counts generally lead to decreased decoding accuracy, while increasing the number of layers exhibits a diminishing returns effect. The number of layers, however, has no significant impact on perplexity (PPL). Therefore, in this work, we primarily adopt m=30 for our experiments. Table 9 : The result of Worldcup sampling across different number of layers m . Model C4 Dataset layers m=5 layers m=10 layers m=20 layers m=30 layers m=50 + Watermark Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Llama3-8B-Base + WorldCup ( k=1 ) 0.954 0.931 5.688 0.011 0.998 0.972 6.188 0.011 0.995 0.985 6.625 0.011 1.000 0.987 6.563 0.011 0.998 0.984 7.063 0.011 + WorldCup ( k=2 ) 0.995 0.990 9.250 0.011 1.000 0.994 9.250 0.012 0.997 0.991 10.13 0.012 1.000 0.995 10.94 0.012 0.997 0.985 16.00 0.012 Gemma2-9B-Base + WorldCup ( k=1 ) 0.932 0.883 5.719 0.011 0.975 0.933 5.938 0.011 0.993 0.957 6.078 0.012 0.997 0.972 6.125 0.013 0.980 0.963 6.031 0.012 + WorldCup ( k=2 ) 0.997 0.978 8.250 0.011 0.993 0.978 8.500 0.011 0.993 0.983 8.375 0.012 0.997 0.984 9.500 0.012 1.000 0.977 10.94 0.012 In addition, we examine the effect of the number of leaves N , as reported in Table 10 . The results indicate that setting N=2 provides the best trade-off between message decoding accuracy and text quality. Table 10 : The result of Worldcup sampling across different number of leaves N . Model C4 Dataset leaves N=1 leaves N=2 leaves N=3 leaves N=4 + Watermark Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Time (s) \downarrow Llama3-8B-Base + WorldCup (16 bit) 0.670 0.495 5.500 0.011 0.998 0.985 6.719 0.011 1.000 0.998 15.38 0.011 1.000 1.000 53.75 0.012 + WorldCup (24 bit) 0.672 0.492 5.500 0.012 0.997 0.958 6.781 0.012 1.000 0.988 15.88 0.012 1.000 0.994 60.00 0.012 + WorldCup (32 bit) 0.670 0.493 5.500 0.012 0.995 0.925 6.875 0.013 1.000 0.972 15.63 0.013 1.000 0.991 61.75 0.013 + WorldCup (48 bit) 0.668 0.469 5.469 0.014 0.990 0.875 6.547 0.013 1.000 0.933 16.25 0.014 1.000 0.968 58.00 0.014 I.5 Different Activation Function We evaluate three activation functions in Eq. 6 , namely ReLU, Sigmoid, and Tanh. The results are reported in Table 11 . Overall, the choice of activation function leads to comparable performance trends, and all variants exhibit a consistent trade-off between bit-level detection accuracy and text quality measured by perplexity. Specifically, ReLU exhibits relatively stronger detection performance due to its linear behavior for positive inputs; however, this gain is accompanied by a noticeable degradation in text quality. In contrast, Tanh and Sigmoid introduce nonlinear saturation, resulting in more conservative modulation and better preservation of generation quality. Since entropy values are non-negative, Tanh effectively normalizes them into the interval [0,1] , whereas Sigmoid maps them into a narrower range of [0.5,1] , which limits the dynamic range of modulation. Based on these considerations, we adopt the Tanh activation function in our method. Combined with a fixed scaling factor \alpha=1.2 , this choice provides a balanced trade-off between detection accuracy and text quality, while allowing stable and interpretable control of watermark strength in practice. Table 11 : The comparison of different activation functions on C4 dataset. Model C4 Dataset 16 bits 24 bits 32 bits 48 bits 64 bits + Watermark Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Llama3-8B-Base + WorldCup (ReLU, \alpha=0.8 ) 1.000 0.989 10.75 1.000 0.974 10.94 1.000 0.959 10.56 0.997 0.921 10.50 0.995 0.885 11.06 + WorldCup (ReLU, \alpha=1 ) 1.000 0.991 11.44 1.000 0.987 11.72 1.000 0.971 12.00 1.000 0.936 12.19 0.997 0.899 12.38 + WorldCup (Sigmoid, \alpha=1 ) 0.997 0.988 9.719 1.000 0.970 9.813 1.000 0.956 9.500 0.990 0.913 9.719 0.997 0.870 9.500 + WorldCup (Sigmoid, \alpha=1.2 ) 1.000 0.990 11.06 0.997 0.979 11.25 1.000 0.963 11.44 1.000 0.925 11.63 1.000 0.891 11.44 + WorldCup (Tanh, \alpha=1 ) 1.000 0.984 9.500 0.995 0.966 9.813 0.995 0.951 9.625 0.992 0.914 9.875 0.997 0.871 9.625 + WorldCup (Tanh, \alpha=1.2 ) 0.997 0.991 10.84 0.995 0.976 10.75 0.995 0.956 10.25 0.997 0.918 10.94 0.995 0.889 10.66 Gemma2-9B-Base + WorldCup (ReLU, \alpha=0.8 ) 0.997 0.960 8.375 0.993 0.930 8.500 0.990 0.911 8.188 0.985 0.871 8.375 0.983 0.822 8.000 + WorldCup (ReLU, \alpha=1 ) 0.987 0.967 9.250 0.992 0.955 9.063 0.990 0.924 9.125 0.992 0.886 9.406 0.982 0.841 9.188 + WorldCup (Sigmoid, \alpha=1 ) 0.990 0.968 8.375 0.992 0.940 7.750 0.992 0.914 7.938 0.977 0.867 8.375 0.980 0.827 8.250 + WorldCup (Sigmoid, \alpha=1.2 ) 1.000 0.968 9.406 0.993 0.957 9.313 0.992 0.924 9.313 0.990 0.879 9.813 0.987 0.835 9.625 + WorldCup (Tanh, \alpha=1 ) 0.995 0.962 7.750 0.995 0.934 7.688 0.992 0.901 7.875 0.977 0.853 7.750 0.972 0.813 7.875 + WorldCup (Tanh, \alpha=1.2 ) 0.993 0.970 8.844 0.990 0.941 8.500 0.982 0.916 8.688 0.992 0.869 8.500 0.988 0.826 8.500 I.6 Key Generation Hyperparameters During watermark generation, both the no_repeat_ngram_size and the window size significantly affect multi-bit watermark performance, since the random seed for embedding is derived from hashing tokens within a context window. If text diversity is too low (e.g., \textit{no\_repeat\_ngram\_size}=0 ), identical seeds are repeatedly used, leading to uneven bit allocation across tokens and poor decoding accuracy (Table 12 ). However, increasing diversity does not always help: setting \textit{no\_repeat\_ngram\_size}=1 severely degrades fluency, as reflected by high perplexity. Balancing decoding accuracy and text quality, we set \textit{no\_repeat\_ngram\_size}=4 . Moreover, window size also influences robustness by determining the extent of context used in hashing. Empirically, larger windows reduce robustness under attacks (Table 13 and Table 14 ). We therefore use a window size of 2 to achieve a favorable trade-off between robustness and fluency. Table 12 : The effect of different no_repeat_ngram_size on multi-bit watermarking methods. Watermark Llama3-8B-Base no_repeat_ngram_size=0 no_repeat_ngram_size=1 no_repeat_ngram_size=2 no_repeat_ngram_size=3 no_repeat_ngram_size=4 Diversity \uparrow Bit Acc \uparrow Perplexity \downarrow Diversity \uparrow Bit Acc \uparrow Perplexity \downarrow Diversity \uparrow Bit Acc \uparrow Perplexity \downarrow Diversity \uparrow Bit Acc \uparrow Perplexity \downarrow Diversity \uparrow Bit Acc \uparrow Perplexity \downarrow BiMark 5.356 0.947 6.375 20.00 1.000 45.25 18.03 0.993 15.63 8.890 0.994 12.56 7.804 0.992 11.81 WorldCup ( k=1 ) 5.036 0.885 3.070 20.00 1.000 34.25 17.88 0.987 9.500 7.715 0.981 7.344 7.202 0.978 6.625 WorldCup ( k=2 ) 6.110 0.947 7.750 20.00 1.000 62.75 18.92 0.996 18.88 8.794 0.996 15.38 7.929 0.994 14.25 Table 13 : The effect of different window size c across our framework WorldCup on LLaMA3-8B-Base. Watermark Llama3-8B-Base window_size c=1 window_size c=2 window_size c=3 window_size c=4 + Attack TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow WorldCup ( k=1 ) 0.995 0.005 0.995 0.951 1.000 0.000 1.000 0.983 0.995 0.000 0.997 0.984 0.990 0.000 0.995 0.987 + Word-D (ratio=0.2) 0.985 0.020 0.983 0.907 0.960 0.000 0.980 0.933 0.930 0.005 0.961 0.911 0.895 0.020 0.935 0.873 + Word-S-DICT (ratio=0.2) 0.965 0.025 0.970 0.895 0.930 0.005 0.961 0.908 0.960 0.065 0.948 0.868 0.845 0.070 0.883 0.842 + Word-S-BERT (ratio=0.3) 0.970 0.025 0.972 0.893 0.950 0.000 0.974 0.924 0.930 0.030 0.949 0.906 0.900 0.020 0.938 0.871 + Copy-Paste (CP-3-30%) 0.880 0.025 0.924 0.886 0.910 0.000 0.953 0.926 0.925 0.015 0.954 0.931 0.885 0.015 0.932 0.913 + Translation (en-zh) 0.990 0.975 0.668 0.605 0.990 0.960 0.671 0.603 0.985 0.955 0.670 0.563 0.980 0.965 0.666 0.538 + Rephrase (GPT-4o) 0.870 0.250 0.821 0.742 0.735 0.325 0.714 0.729 0.885 0.630 0.704 0.657 0.805 0.565 0.679 0.634 + Dipper-1 (lex=0, order=0) 0.945 0.020 0.962 0.881 0.940 0.035 0.952 0.904 0.910 0.045 0.931 0.887 0.885 0.020 0.929 0.873 + Dipper-2 (lex=20, Dipper=20) 0.895 0.110 0.893 0.793 0.865 0.180 0.846 0.772 0.765 0.130 0.807 0.738 0.770 0.295 0.746 0.698 WorldCup ( k=2 ) 0.995 0.020 0.988 0.974 1.000 0.000 1.000 0.991 1.000 0.000 1.000 0.993 1.000 0.000 1.000 0.994 + Word-D (ratio=0.2) 0.995 0.055 0.971 0.941 0.975 0.000 0.987 0.949 0.975 0.010 0.982 0.933 0.980 0.035 0.973 0.905 + Word-S-DICT (ratio=0.2) 0.980 0.050 0.966 0.920 0.975 0.020 0.977 0.942 0.940 0.050 0.945 0.909 0.925 0.035 0.944 0.888 + Word-S-BERT (ratio=0.3) 0.995 0.055 0.971 0.923 0.980 0.035 0.973 0.940 0.945 0.045 0.950 0.922 0.975 0.060 0.958 0.897 + Copy-Paste (CP-3-30%) 0.920 0.055 0.932 0.909 0.925 0.045 0.939 0.940 0.950 0.010 0.969 0.955 0.900 0.010 0.942 0.936 + Translation (en-zh) 0.850 0.625 0.687 0.676 0.990 0.960 0.671 0.636 0.960 0.905 0.670 0.589 0.995 0.975 0.670 0.582 + Rephrase (GPT-4o) 0.725 0.080 0.803 0.783 0.850 0.330 0.780 0.735 0.830 0.425 0.736 0.693 0.810 0.570 0.681 0.666 + Dipper-1 (lex=0, order=0) 0.970 0.060 0.956 0.890 0.970 0.035 0.968 0.911 0.945 0.055 0.945 0.903 0.860 0.035 0.908 0.883 + Dipper-2 (lex=20, Dipper=20) 0.860 0.080 0.887 0.798 0.825 0.165 0.829 0.790 0.845 0.390 0.756 0.719 0.765 0.330 0.730 0.693 Table 14 : The effect of different window size c across our framework WorldCup on Gemma2-9B-Base. Watermark Gemma2-9B-Base window_size c=1 window_size c=2 window_size c=3 window_size c=4 + Attack TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow TPR \uparrow FPR \downarrow F1 \uparrow Bit Acc \uparrow WorldCup ( k=1 ) 0.995 0.015 0.990 0.909 0.995 0.010 0.993 0.943 0.995 0.000 0.997 0.973 1.000 0.005 0.998 0.974 + Word-D (ratio=0.2) 0.955 0.030 0.962 0.872 0.875 0.085 0.893 0.868 0.860 0.050 0.901 0.848 0.830 0.105 0.858 0.813 + Word-S-DICT (ratio=0.2) 0.945 0.075 0.936 0.853 0.890 0.165 0.866 0.832 0.945 0.235 0.867 0.827 0.810 0.170 0.818 0.784 + Word-S-BERT (ratio=0.3) 0.930 0.025 0.951 0.851 0.925 0.140 0.896 0.843 0.830 0.055 0.881 0.847 0.815 0.150 0.830 0.828 + Copy-Paste (CP-3-30%) 0.875 0.030 0.919 0.836 0.880 0.030 0.921 0.874 0.885 0.025 0.927 0.879 0.885 0.040 0.919 0.884 + Translation (en-zh) 0.990 0.970 0.669 0.590 1.000 0.990 0.669 0.580 0.995 0.990 0.667 0.552 0.995 1.000 0.664 0.516 + Rephrase (GPT-4o) 0.790 0.280 0.763 0.712 0.760 0.335 0.726 0.701 0.800 0.460 0.708 0.681 0.920 0.750 0.689 0.647 + Dipper-1 (lex=0, order=0) 0.925 0.070 0.927 0.781 0.825 0.030 0.889 0.795 0.819 0.055 0.874 0.817 0.760 0.070 0.831 0.798 + Dipper-2 (lex=20, Dipper=20) 0.840 0.140 0.848 0.716 0.790 0.175 0.804 0.704 0.845 0.480 0.727 0.688 0.880 0.640 0.698 0.655 WorldCup ( k=2 ) 0.970 0.020 0.975 0.947 0.995 0.000 0.997 0.968 0.995 0.000 0.997 0.983 0.995 0.000 0.997 0.984 + Word-D (ratio=0.2) 0.950 0.085 0.934 0.909 0.935 0.045 0.944 0.905 0.915 0.060 0.927 0.887 0.905 0.065 0.919 0.858 + Word-S-DICT (ratio=0.2) 0.905 0.060 0.921 0.877 0.845 0.045 0.894 0.882 0.860 0.060 0.896 0.869 0.855 0.110 0.870 0.824 + Word-S-BERT (ratio=0.3) 0.910 0.060 0.924 0.896 0.850 0.045 0.897 0.878 0.865 0.060 0.899 0.867 0.815 0.110 0.847 0.830 + Copy-Paste (CP-3-30%) 0.910 0.050 0.929 0.886 0.895 0.010 0.939 0.900 0.895 0.015 0.937 0.909 0.910 0.010 0.948 0.905 + Translation (en-zh) 0.995 1.000 0.664 0.641 0.995 0.995 0.666 0.602 0.990 0.970 0.669 0.588 0.975 0.945 0.668 0.537 + Rephrase (GPT-4o) 0.795 0.255 0.776 0.756 0.825 0.575 0.688 0.709 0.935 0.795 0.685 0.690 0.965 0.875 0.680 0.634 + Dipper-1 (lex=0, order=0) 0.905 0.165 0.874 0.805 0.825 0.045 0.882 0.825 0.795 0.065 0.855 0.803 0.785 0.095 0.835 0.775 + Dipper-2 (lex=20, Dipper=20) 0.890 0.290 0.817 0.715 0.865 0.400 0.764 0.710 0.840 0.545 0.704 0.649 0.875 0.645 0.694 0.626 I.7 Counting-based Decoding vs. Confidence-aware Decoding To rigorously demonstrate that the proposed confidence-aware decoding outperforms conventional counting-based decoding, we compare the decoding accuracy of WorldCup under both strategies, as reported in Table 15 . The results clearly show that confidence-aware decoding consistently achieves higher accuracy across different embedded message bit lengths. This improvement fundamentally stems from its ability to mitigate the adverse influence of low-entropy tokens and instead rely more heavily on high-entropy tokens, which provide more reliable statistical evidence for decoding. This observation is also consistent with conclusions drawn in prior zero-bit watermarking studies, such as EWD ( 40 ) , further validating the effectiveness of entropy-aware decoding strategies. Table 15 : The comparison of counting-based decoding and confidence-aware decoding. Model C4 Dataset 16 bits 24 bits 32 bits 48 bits 64 bits + Watermark Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Best F1 \uparrow Bit Acc \uparrow PPL \downarrow Llama3-8B-Base + WorldCup ( k=1 , counting-based) 0.982 0.971 6.688 0.985 0.933 6.625 0.966 0.897 6.969 0.965 0.844 6.750 0.925 0.771 6.594 + WorldCup ( k=1 , confidence-aware) 0.998 0.984 6.625 0.997 0.961 6.750 0.995 0.930 7.000 0.992 0.875 6.563 0.990 0.813 6.547 + WorldCup ( k=2 , counting-based) 0.931 0.978 11.00 0.926 0.959 11.16 0.908 0.925 11.06 0.883 0.886 11.44 0.889 0.843 11.44 + WorldCup ( k=2 , confidence-aware) 1.000 0.990 11.00 1.000 0.982 11.25 1.000 0.962 11.16 1.000 0.925 11.44 1.000 0.887 11.16 I.8 Algorithm Algorithm 2 WorldCup Message Decoding (Confidence-Aware) 1: Input: watermarking key \xi and hash seed r_{t} , total message length b , 2^{k} -ray, g -value function families \{\mathbf{g}_{j},\mathbf{\bar{g}}_{j}\}_{j=1}^{k} 2: Compute number of message symbols B\leftarrow b/k 3: Initialize decoded message symbols \hat{\mathbf{m}}\in\{0,\ldots,2^{k}-1\}^{B} 4: Use (r_{t},\xi) to map each token y_{t} to a message position p_{t}\in\{0,\ldots,B-1\} 5: Group tokens by message position: \mathcal{S}_{p}=\{y_{t}\mid p_{t}=p\} 6: for p=0\ \text{to}\ B-1 do 7: for j=1\ \text{to}\ k do 8: Compute confidence scores by averaging over the group: s_{j}^{p}\leftarrow\frac{1}{|\mathcal{S}_{p}|}\sum_{y_{t}\in\mathcal{S}_{p}}g_{j}(y_{t}),\ \bar{s}_{j}^{p}\leftarrow\frac{1}{|\mathcal{S}_{p}|}\sum_{y_{t}\in\mathcal{S}_{p}}\bar{g}_{j}(y_{t}) 9: end for 10: Decode the 2^{k} -ray message symbol: \hat{m}_{p}\leftarrow\sum_{j=1}^{k}2^{k-j}\,\mathbb{I}\!\left(s_{j}^{p}<\bar{s}_{j}^{p}\right) 11: end for 12: Assemble decoded symbols into message sequence: \hat{\mathbf{m}}\leftarrow(\hat{m}_{0},\hat{m}_{1},\ldots,\hat{m}_{B-1}) 13: Return decoded message symbols \hat{\mathbf{m}} Appendix J Case Study J.1 Llama3-8B-Base Examples Prompt (C4 Dataset) Applications for entrance in the Fall of 2019 are now open. We will begin to evaluate applications on January 15, 2019. If your application is not Natural Text (C4 Dataset) received by then, it may not be evaluated. Please note that we do not typically admit for Spring or Summer admission. Offers are usually made between February and April 15. All applicants are considered for department support via research assistantships, teachings assistantships, and merit-based scholarships. Virtually all graduate students are supported for the academic year and in many cases during the summer term as well. Once the Graduate Admissions Committee has made a final decision on your application they will notify you via the email address that is on your application. There is no minimum GRE or GPA requirement. However, to be competitive for assistantships and scholarships, students need a GPA of at least 3.0, a combined verbal and quantitative score on the GRE of at least 300, and an analytical writing score on the GRE of at least 3.5. The average GRE score for students that were accepted into our program for Fall 2018 was 316. Unwatermarked Text (Llama3-8B-Base) complete by that date we will not be able to guarantee you an interview. Click here to begin an application. Please visit the requirements tab above for details about how to complete an application. The deadline for submitting applications for Fall, 2019 is March 31, 2019. A maximum of two interviews a week will be held every Tuesday at 1:00 p.m. and 3:00 p.m., and Wednesday at 9:00 a.m. and 1:00p.m. You may not request interview dates. There is no interview in the Spring. Interviewers are assigned by the Office of Admission. You will not be interviewed by anyone that you request for an interview. Watermarked Text (Message: 1101000010110000 Bit Accuracy: 1.0 k=2) complete or received before January 15, you will be placed in the second round of applications. Please be aware there is limited or no amount of spots for second round applications. Please be aware that all our students for the U.S. and Canada are in class by April 1, for non-residents this is 2 weeks later. We do not keep our spots open for the full year. If you cannot get into Canada by April 1 you cannot join our Program. We will not make arrangements for you to join the Program at a later date. For spots that become available, even if we have already started the Program, only students who have the ability to be in Canada by April 13 will be admitted. Prompt (C4 Dataset) Independent Bank Corp. (nasdaq global select market:INDB), parent of Rockland Trust Company, today announced 2018 fourth quarter net income of Natural Text (C4 Dataset) $29.9 million, or $1.07 per diluted share, compared to net income of $33.0 million, or $1.20 per diluted share, reported in the prior quarter of 2018. Excluding merger and acquisition expenses incurred in the fourth and third quarters of 2018 related to the MNB Bancorp (MNB) merger which closed in November 2018 and the Blue Hills Bancorp (BHB) merger announced on September 20, 2018, operating net income was $35.9 million, or $1.29 per diluted share during the fourth quarter of 2018 compared to $34.9 million, or $1.27 per diluted share during the third quarter of 2018. Full year net income was $121.6 million, or $4.40 on a diluted earnings per share basis, an increase of $34.4 million, or 39.5%, as compared to the prior year. In addition, full year operating net income was $129.8. Unwatermarked Text (Llama3-8B-Base) $35.8 million, or $0.78 per diluted share, an increase of 2.7% and 3.3% from the $34.9 million, or $1.02 per diluted share, reported for the 2017 fourth quarter. For the 2018 full year, net income increased 20.1% to $137.4 million, or $3.03 per diluted share, compared to $114.3 million, or $2.88 per diluted share, for the 2018 year ended December 31, 2017. The 2018 fourth quarters and full year included $5.6 million in total credit loss expense, compared to the corresponding prior year amounts of $4.0 million and $16.6 million. The 2017 fourth quarters and full years included $3.6 million and $6.3 million, respectively, in additional expense for the tax effect of deferred tax liabilities associated with the impact of the change in U.S. federal corporate tax rates from 35% to 21% as a result of the Tax Cuts and Jobs Act. “During the quarter we completed a successful acquisition of the 17 branches and other deposits from Eastern Bank. Watermarked Text (Message: 001111111010010000101010 Bit Accuracy: 0.75 k=2) $26.6 million, or $0.79 per share, compared to $23.5 million, or $1.40 per share, in the prior year’s fourth quarter. For the full year 2018, net income was $101.7 million, or $3.06 per share, compared with $85.2 million, or $5.29 per share, in 2017. The Company’s 2018 performance was impacted by the non-cash charge of $10.7 million, which was added back to net income in the calculation of earnings per share, to establish a valuation allowance against its deferred tax assets. This charge was partially offset by a $6.1 million, or $10.1 million on an after-tax basis, increase in net interest income due to lower interest expense. Pre-tax, pre-provision net revenue increased 15.1%, driven by growth in loans and deposits, higher non-interest income and lower non-interest expense. J.2 Gemma2-9B-Base Examples Prompt (C4 Dataset) Our no. 1 sneaker of 2014 just got a serious upgrade in the form of the adidas Pure Boost ZG Prime. We first Natural Text (C4 Dataset) previewed this sneaker in January, and now the next-gen of Pure Boost has arrived. This model mixes a full-length Boost midsole with a layer of EVA on the outsole to produce what adidas calls a f̈loating sensation,ḧence the Zero Gravity’s; (ZG) part of its name. That’s not the only key feature here, as this version of the Pure Boost ZG also features a Primeknit upper for a cozy and adaptive sock-like fit. It’s then finished off with leather tabs at the tongue and heel, adding a premium touch to an otherwise sporty silhouette. If for some reason this Primeknit variation isn’t quite your cup of tea—or out of your budget—there’s also a less expensive mesh-based version dropping this summer. The adidas Pure Boost ZG Prime is now available in black. Unwatermarked Text (Gemma2-9B-Base) laid eyes on the Adidas PureBoost ZG Prime at the recent 2015 PGA Show where the model made its debut on the feet of former World No. 1 and 3-time Major champion, Ernie Els. For the 2016 season, Ernie Els has teamed up with Adidas Golf and will don the PureBoost ZG on his feet during the Major season. It’s no secret that adidas Pure Boost technology is the best when it comes to running. The adidas PureBoost ZG and PureBoost ZG Primes boast a flexible woven upper that provides a sock-like fit with minimal movement. adidas Pureboost ZG Prime are made with Primeknit upper, featuring woven technology for flexible fit. adidas Pure Boost ZG. Adidas Pure Boost ZG/Primes are engineered with Primeknit upper, featuring flexible material and delivering comfortable fit. adidas has now released the PureBoost Z and ZG Primes. Adidas PureBoost ZG will also launch this winter in six color options. adidas just released the PureBoost Prime, and now the ZG model is set to make its debut in the near future. Watermarked Text (Message: 1101101011101010 Bit Accuracy: 1.0 k=2) saw the Pure Boost ZG back in July, and one of our editors was lucky enough to try them out during the 2015 World Championships in Beijing, China. The adidas Pure Boost Zg Prime is an update to the most popular running shoe of 2004, the Pure Boost Zg. The Zg Prime has a more streamlined upper and a wider base for a more comfortable fit. The new upper is made of 2 layers of a breathable material, with a 3rd layer of support at the medial side of the shoe. The adidas pure boost zg prime is the latest in a long line of running shoes from adidas. It features a Boost midsole and a Primeknit upper. The shoe also features an EVA midsole for cushioning and a Torsion System for stability. The adidas Zg Prime is a running shoe that features an Air Max unit, a Boost midsole, and a Primeknt upper. The shoe is available in two colorways: Black/White/Grey and White/Black/Grey. The black version features a suede upper with a leather overlay, while the white version features a mesh upper with a leather liner. Both shoes feature a Boost midsole with an Air Max unit. Prompt (C4 Dataset) A child in Florida has become the first person to die of the flu this season, according to state health officials. State epidemiologists say the Natural Text (C4 Dataset) child had not been vaccinated and was otherwise healthy before getting sick with the flu. The child, who tested positive for influenza B, died sometime during the week of Sept. 30, although privacy concerns prevent officials from saying exactly where, CBS affiliate WTSP-TV reports. Last flu season, 183 children in the U.S. died from flu or flu-related causes. That’s the most since the Centers for Disease Control and Prevention (CDC) began keeping these records in 2004. Overall, an estimated 80,000 Americans died from flu last season. CBS News medical contributor Dr. Tara Narula says this latest news should be a wake-up call to parents to get their children vaccinated. ”What this is a strong clear message to parents about the importance of vaccination,” she told ”CBS This Morning.” ”This vaccine is safe. It is the most effective tool we have. And we know of the pediatric deaths last year, 80 percent were in kids who were unvaccinated. Unwatermarked Text (Gemma2-9B-Base) child died from influenza B. That’s different from the type of flu A that caused death and illness in children last year. The CDC says 17 children died of the flu in the 2009-10 flu season, 11 of them from the H1N1 strain, or swine flu. Last year 18 children in 10 states died of swine flu, according to the AAP. That’s in addition to eight children who died from the flu last year. Watermarked Text (Message: 011001110110011011100110 Bit Accuracy: 0.958 k=2) child died of flu complications, but did not specify the age of the child or where in the state they lived. It’s unclear if the child had been vaccinated against the flu. A spokesperson for the Florida Department of Health said the state does not release information about the individual flu cases. The U.S. Centers for Disease Control and Prevention currently reports that the flu has spread to 46 states, and has killed at least 11 people so far this season. According to the CDC, the flu has hit people hardest 50 to 64 years old. The CDC also reports that in the last week, 45 people out of every 100,000 have sought medical treatment for flu-like symptoms. That’s up from 26 the week before. Health officials say this year’s flu strain is particularly. The CDC says 75 percent of people who die from the flu each year are 65 or older. Some experts say this year’ll be flu season. But they say there’s still time to get vaccinated against the flu, and they urge people to do so.