It’s easier than you might think to bargain away the right to challenge patent validity at the U.S. Patent and Trademark Office.

The U.S. Court of Appeals for the Federal Circuit on Thursday upheld San Francisco U.S. Magistrate Judge Elizabeth Laporte’s order last year compelling MerchSource LLC, also known as Sharper Image, to withdraw its administrative challenges to three patents on virtual reality headsets.