Satisfying ternary permutation constraints by multiple linear orders or phylogenetic trees

Abstract

A ternary permutation constraint satisfaction problem (CSP) is specified by a subset of the symmetric group S3. An instance of such a problem consists of a set of variables Vand a set of constraints C, where each constraint is an ordered triple of distinct elements from V. The goal is to construct a linear ordering αon Vsuch that, for each constraint (a, b, c) ∈C, the ordering of a,b,cinduced by αis in . Excluding symmetries and trivial cases there are eleven such problems, and their complexity is well known. Here we consider the variant of the problem, denoted 2-, where we are allowed to construct two linear orders αand βand each constraint needs to be satisfied by at least one of the two. We give a full complexity classification of all eleven 2-problems, observing that in the switch from one to two linear orders the complexity landscape changes quite abruptly and that hardness proofs become rather intricate. To establish the proofs, we use several computer-aided searches. Each such search returns a small instance with a unique solution for a given problem. We then focus on one of the eleven problems in particular, which is closely related to the 2-Caterpillar Compatibilityproblem in the phylogenetics literature. We show that this particular CSP remains hard on three linear orders, and also in the biologically relevant case when we swap three linear orders for three phylogenetic trees, yielding the 3-Tree Compatibilityproblem. Lastly, we give extremal results concerning the minimum number of trees required, in the worst case, to satisfy a set of rooted triplet constraints on nleaf labels.