# Balancing Statistics After Opener Bids One of a Suit

by Matthew Kidd

Recently I have been posting Maritha Pottenger’s many teaching handouts. She asked that I review them first so I’ve been acting as an editor which entails the usual forgettable activities. But occasionally a real question arises, usually one involving probability or numbers such as average HCP or suit length that can be refined by simulation. When I reviewed her Balancing handout I thought most of the HCP ranges in her opening paragraph were slightly off. But I wasn’t sure. So I dusted off Thomas Andrews’ Deal program and ran a simulation.

The specific scenario is balancing after opener bids one of suit, i.e. 1♣/1/1/1♠ – Pass – Pass to the balancer. The simulation results assume opener is in first or second seat and plays a strong notrump (15-17). Full details of the simulation, including the Tcl code, are presented later. The figure below summarizes the results.

The data points show the average strength for each of the other three hands when the balancer holds a given number of high card points (HCPs). The total HCP strength of each partnership is also shown. The shaded areas show one standard deviation about the average. Each column of data points is based on a simulation of 10,000 deals except the 0-2 HCP columns which are based on only 1,000 deals for performance reasons detailed later. The simulation assumes an aggressive open-responder bidding style meant to reflect what is typically encountered in the open pairs game at an ACBL regional. The dotted lines are the result for a second simulation using a conservative opener-responder bidding style. The conservative simulation result is not shown for responder because it is almost identical to the result from the aggressive simulation.

Parity in strength for each side on average is achieved when balancer has just about 9 HCP. Bidding books typically recommend balancing with 8 HCP hands and many 7 HCP hands. At 7 HCP, balancer’s side is outgunned 21.7 to 18.3 HCP on average but has 20+ HCP 30% of the time, enough to make competing worthwhile even if it only manages to push opener’s side up a level. Balancer’s side also has the advantage of having values in both hands whereas opener has nearly all the resources for his side and may be hard pressed to reach dummy once, let alone twice.

Responder’s values are rather insensitive to balancer’s values, dropping only 1.6 HCP over a 20 HCP range. Beyond 12 HCP in balancer’s hand, his partner’s expected strength falls sharply, about 0.64 HCP for every extra HCP in balancer’s hand. Therefore, balancer should not get unduly excited about hands bigger than 12 HCP, instead saving his enthusiasm for hands closer to 12 HCP than to 7 HCP because it is in the 7–12 HCP range that the partscore battle swings quickest towards balancer’s favor.

In the conservative bidding scenario, opener’s extra average strength of up to 0.4 HCP primarily comes at the expense of balancer’s partner. The effect isn’t very noticeable until balancer has 11+ HCP.

## Distribution of balancer’s HCP

Balancer is not equally likely to have each number of HCPs. The histogram at the right shows balancer’s HCP for 50,000 deals. On average balancer will have 12.4 HCP and will hold 7+ HCP over 95% of the time and 8+ HCP almost 92% of the time. (In the conservative opener-responder scenario, balancer only averages 12.0 HCP.) 60% of the time, balancer will be in the 10–15 HCP range.

Things are different if opener is in second seat because the balancer then passed initially, limiting his hand. In this case we are drawing only from the left half of the distribution. The balancer will rate to have about 9 HCP cards on average.

## Distribution of opener’s HCP

Unlike balancer’s distribution, opener’s distribution is far from Gaussian (bell shaped). About 50% of the time, opener has a minimal 11-14 HCP hand. Another 40% of the time, opener is in the 15-19 HCP range. The near uniformity of the 15-19 bars is due to the suppression of hands that open would open 1NT whereas with 18 or 19 HCP opener would bid at the one level, planning to rebid 2NT. Incidentally, if responder balances, a rebid of 1NT by opener shows the 18-19 HCP hand that would normally be shown with a 2NT rebid in an uncontested auction. With 11-14 HCP, opener should pass the balancing bid. If a 1NT rebid is passed out, opener will often be very happy not to be in 2NT.

The figure above shows three cases: all deals, those when balancer has 7+ HCP (most likely will balance), and when balancer has 12 HCP, the most common case. Opener’s distribution does not change much in these three scenarios.

## Distributions as a function of balancer’s strength

The plot below gives a closer look at what is summarized by the shaded regions in the initial figure. Each row represents a range of balancer’s HCP as indicated by the text at the far left of each row. The corresponding HCP distributions, based on 10,000 deals per row, are shown for opener + responder, opener, responder, balancer’s partner, and balancer + partner. The vertical axis in each plot is the percent of hands with a given HCP value. All plots in a single column have the same x-axis range, the one shown by the bottom plot. The distributions in the first and last columns are mirror images since the total HCP in the deck is fixed at 40.

## Simulation details

The simulation code appears below. Code comments are in green. Program lines that end with a backslash continue on to the next line, the Tcl line continuation standard.

This is a fairly complicated Tcl program and it uses some of the Deal program’s advanced features. Nonetheless it proceeds in a fairly straightforward manner. First some flags and parameters are set, e.g. set dump_hcp 1. The first two flags control what is printed on STDOUT usually for redirection to a file which can then be read by a program like Matlab to produce quality figures and plots.

The next five parameters control the style of the opener-responder partnership. For the aggressive bidding simulation the first four of these are set to 1 (true). For the conservative bidding scenario, these are set to 0 (false). You can adjust them as you fit. The aggressive settings mean:

1. Opener will open all 11 HCP hands containing two aces and a king (2½ quick tricks) at the one level, regardless of whether the king is suited with an ace.
2. Opener will open all 12 HCP hands, even flat ones, at the one level; otherwise opener will only open a 12 HCP with a five card or longer suit.
3. Opener will open all 11 HCP with a six card or longer suit at the one level; otherwise opener is assumed to preempt or pass (don’t care which for the simulation).
4. Responder may bid with fewer than 6 HCP in an attempt to improve the contract. The specific scenarios allowed in the code are 4-4 in the majors with any 5 HCP hand, 4 cards in one major with AJ (aces are slightly undervalued at 4 HCP), or 5-5 in the majors with just an ace. If this flag is set to zero, responder must have 6+ HCP to respond.

Irrespective of #4 above, responder may be permitted to make Weak Jump Shift responses with a six card or longer suit and 0-5 HCP by enabling the wsj_flag. This option had little effect on the results of either the aggressive or conservative simulation and was therefore turned off when generating the results presented above.

The putative overcaller settings are discussed later.

After creating a few statistics objects for calculating means and standard deviations, the main business begins in the block. The Deal program feeds random deals to the main {…} block whose primary job it is to reject deals not meeting the simulation criterion. The end of this block tracks some statistics and other information for the deals that do meet the criterion. Fast rejection speeds up the simulation so it is best to order the rejection criterion with this in mind.

The first check is make sure balancer’s HCP falls with a desired range specified by \$b_min and \$b_max. If you want to consider all balancer’s hands you can set these to 0 and 37 respectively, though 30 is also fine for the upper limit because opener needs values. You might notice that the set b_min and set b_max statements are commented out in the code. That is because their values can be specified via the command line, allowing easy iteration over many cases using a shell script or a Perl program, as discussed later. You may uncomment them if you simply want to use the code as presented.

Next we figure out if opener will open at the one level. We immediately reject hands with fewer than 11 HCP or greater than 22 HCP. It is not relevant whether opener will pass or preempt with weak hand. It is enough to know he will not open at the one level. Some will argue that with 10 HCP, opener may start at the one level with a long good suit or a two-suited “Rule of 20” hand. But these hands are rare and including them will barely change the simulation results. Still, you are free to make these changes yourself.

The simple HCP based rejection is followed by more complicated opener rejection criterion. First we eliminate the cases where the opening bid would be 1NT or 2NT. Note the that the balanced shapefunc includes 5-3-3-2 hands when the five card suit is a minor but not when it is a major. This means we are simulating an opener who does not open notrump with a five card major. Change this section of code if you want to allow for notrump with five card majors or off-shape notrump, e.g. 6-3-2-2 with a six card minor. After the notrump rejection, we deal with 11 HCP and 12 HCP hands that depend on our opener aggressiveness settings.

If opener’s hand is acceptable, check that responder will pass, rejecting the deal if not. This is easier if responder only bids with 6+ HCP, slightly more complicated otherwise.

Having performed the easy rejections, we finally come to the lengthy code to determine if the putative overcaller would have overcalled or doubled in which case the deal must be rejected. This determination requires knowing what call opener made. The call is determined by the shape function code opening_suit which I copied from the Advanced User Guide for the Deal program. Within the function \$c, \$d, \$h, and \$s represent the lengths of the club, diamond, heart, and spades suits respectively. The advantage of a shape function (shapefunc) is that it is very fast. Because the result of a shape function must depend only on the shape of a hand, the Deal program computes the result once for all 560 possible shapes at which point the function becomes nothing more than a reference to a lookup table, a very fast operation. The drawback of shape function is that the result can not be depend on non-shape information. For example, if it is coded to return an opening bid of 1 for a 2=2=5=4 shape, it can never return a bid of 1♣ even though the hand might opened as such if opener is strong enough to want to reverse with a 2 rebid. But as with many small simulation issues, they are unlikely to affect the big picture much.

The bidding rules coded for the putative overcaller are:

1. Overcaller will never pass with 17+ HCP. For our purposes, it doesn’t matter whether he overcalls a suit, overcalls 1NT, doubles, even resorts to a four card overcall due to improper shape for a takeout double and a lack of a stopper in opener’s suit for 1NT.
2. Overcaller will preempt or overcall with 6+ cards in a suit other than opener’s suit and 6+ HCP. The 6 HCP minimum can be adjusted using the \$hcpp parameter in the settings for overcaller.
3. Overcaller will make a 1-level overcall with a 5 card suit and 9+ HCP (adjust with the \$hcp1 parameter).
4. Overcaller will make a 2-level overcall with a 5 card suit and 13+ HCP (adjust with the \$hcp2 parameter).
5. Overcaller will make a takeout double with 14+ adjusted points and will have at least four cards in any unbid majors and two cards in the unbid minor over a 1♣/ opener and at least four cards in the unbid major and 4-3 or longer in the minors over a 1/♠ opener. Adjusted points are defined as HCP + dummy points from shortness in opener’s suit (void = 5, singleton = 3, doubleton = 1). The value of 14 can be adjusted with the \$ptd parameter.
6. Overcaller will make a 1NT overcall with 15-18 HCP and a stopper, defined as ace, king, queen, JTxx, Jxxx, Txxx, or any five cards in opener’s suit.

The code does not determine what bid overcaller would make because it doesn’t matter. All that matters is the deal be rejected if the putative overcaller would not pass. Not needing to be a bidding engine allows for a degree of sloppiness. For example, the no trump overcall criterion may seem sloppy but it isn’t necessary to check for a balanced hand because unbalanced hands with 15-16 HCP will have been rejected earlier by the overcall or takeout double tests, and 17+ HCP hands are simply assumed to find a bid. At worst these rules may reject a 4-4-4-1 with the stiff in a suit different from opener’s suit on the 1NT overcall basis but then again some player may well bid 1NT on such hands or find a four card overcall.

Many refinements to overcaller’s behavior are possible. But in practice even significant adjustments to the current overcaller settings did not change the simulation results much.

Before accepting a deal at the end of the main {…} block, statistics are updated and each hand’s HCP value is optionally printed if the \$dump_hcp flag is set.

When all deals requested have been generated, the block is called once. This displays the collected statistics to the screen, sent to STDERR so as not to end up in a file redirected from STDOUT. If the \$dump_tdt flag is set, it also dumps all the statistics in a single line of tab delimited values. This feature is used when running the code multiple time to collect results for different scenarios, in particular different HCP in balancer’s hand.

## Running the code

To examine a few hands (10 by default) generated by the code, run:

deal -i balancing.tcl -e "set bmin 0; set bmax 30"

The -e switch isn’t well documented in the Deal documentation. I found it mentioned in the documentation for the stack_hand command. It defines Tcl code to be executed at the start of the run. The code here defines balancer’s minimum and maximum HCP. You can skip this switch if you uncomment the set bmin and set bmax lines in the code above.

To collect the statistics used to generate the first figure, tweak the reporting parameters, setting dump_hcp to 0 and dump_tdt to 1. Then run the following commands:

deal -i balancing.tcl -i format/none -e "set bmin 0; set bmax 0" 1000 > balancing-stats.txt
deal -i balancing.tcl -i format/none -e "set bmin 1; set bmax 1" 1000 >> balancing-stats.txt
deal -i balancing.tcl -i format/none -e "set bmin 2; set bmax 2" 1000 >> balancing-stats.txt
deal -i balancing.tcl -i format/none -e "set bmin 3; set bmax 3" 10000 >> balancing-stats.txt
deal -i balancing.tcl -i format/none -e "set bmin 4; set bmax 4" 10000 >> balancing-stats.txt
(…)
deal -i balancing.tcl -i format/none -e "set bmin 20; set bmax 20" 10000 >> balancing-stats.txt

In practice I performed this iterative process using a simple Perl script. The transition from 1000 deal to 10000 deals is discussed in the next section. The addition of -i format/none to the command line suppresses the display of the deals, sending only the final line of statistics to the file.

To collect the data used to generate the distribution of balancer’s HCP, tweak the reporting parameters, setting dump_hcp to 1 and dump_tdt to 0. The run the following command:

deal -i balancing.tcl -i format/none -e "set bmin 0; set bmax 30" 50000 > balancing-50000.txt

Load the resulting tab delimited data into Matlab or Excel and histogram the fourth column, balancer’s HCP.

## Better performance using smartstack

When balancer is limited to very few HCP, the simulation runs slowly because hands with very few HCP are rare. A specified seat will have 0 HCP only once in 275 randomly generated deals. Even then most deals where balancer has 0 HCP will be rejected because responder has enough to bid or the putative overcaller has enough to take action—the 40 HCP in the deck have to go somewhere. The Deal program has to try about 350,000 deals before it finds one that can be accepted.

The Deal program has an alternative deal generation procedure invoked using the smartstack command that works well when the either the HCP range of one hand is rare and/or the allowed shapes for the hand are very restrictive. It works by iterating over the 39 possible patterns and figuring out all the ways the honor cards can be placed to give the HCP range specified. For example, consider the 3 HCP case. For the 4-3-3-3 pattern, the following nine cases exist:

Kxxx xxx xxx xxx
xxxx Kxx xxx xxx
QJxx xxx xxx xxx
Qxxx Jxx xxx xxx
Jxxx Qxx xxx xxx
xxxx QJx xxx xxx
xxxx Qxx Jxx xxx
Jxxx Jxx Jxx xxx
xxxx Jxx Jxx Jxx

Each case comes in multiple varieties depending on how the suit symbols are assigned. The first case has four distinct arrangements of the suits since any suit can be the four card suit. The second case has 12 distinct arrangements of the suits since any suit can be the four card suit and any remaining suit can be the one with the king. Given the possible shapes the user is interested in, each case can be assigned a weight, reflecting its probability of occurring among randomly dealt deals meeting the requested HCP and shape requirements. This in turn allows a single random number to determine a case and an allowed assignment of suits for the case given the shape requirements. Then it is just a matter of randomly assigning small cards as needed to the case selected and then distributing the remaining 39 cards to the other three hands. Voilà, every deal meets the HCP and shape requirements of the one narrowly defined hand.

It can take a bit of time to build up all the cases and work out their relative probabilities. This means smartstack helps most when generating a lot of deals where one hand always has to meet a narrow criterion. The Deal program does not allow more than one hand to be generated via smartstack because the number of cases would become very large.

Here is an example of using it for the case when balancer has 0 HCP:

deal -i balancing.tcl -i format/none -e "set bmin 0; set bmax 0" -I "smartstack east allshapes hcp 0 0" 1000 > balancing-stats.txt

For the balancing simulation above, no restriction is placed on balancer’s shape. However, smartstack still needs to know which shapes are permitted. The trivial shape condition satisfies this need by returning 1 (true) for any given shape.

Even using smartstack, the 0–2 HCP cases for balancer were a bit slow. So I generated only 1000 instead of 10000 deals for those three cases. smartstack didn’t provide as much benefit for the 3–20 HCP cases but was still a small improvement. With smartstack, good statistics were generated for all 21 HCP cases for balancer in about six minutes on a modern laptop (the Deal program is single threaded).

## Get the data

Download a zip file (300 KB) containing the Tcl program above and the data files used to generate the plots. The files balancing-stats-aggressive and balancing-stats-conservative.txt are tab delimited text files giving the HCP average and standard deviation of the south (opener), north (responder), and west (putative overcaller) for each HCP total in balancer’s hand (bmin and bmax columns) in each of the two simulation scenarios. This data is used to generate the first plot. The files with the naming scheme balancing-hcp-##-##-10000 are tab delimited files with the HCP total for south, west, north, and east respectively for each of 10,000 deals where balancer’s HCP total is in the range ##–##. This data is used to generate the remaining plots.