The San Diego Unit Payoff Matrix

by Matthew Kidd
January 7, 2014

Last October I introduced the Payoff Matrix concept and showed results for six years of the La Jolla unit game based on the output generated by the Payoff Matrix software. Here I present similar results for the San Diego unit game and the combination of both unit games over the last two years.

It is most useful to look at regular partnerships that play in enough sessions to generate decent statistics. For the nearly two year period from the start of January 2012 through the end of November 2013, a 10 session minimum cut reduces the number of San Diego unit game partnerships to a manageable 37. Dave Oakley and Dave Walters lead the pack with a 57.55% average, which is very similar to their leading 56.96% average in the La Jolla unit game. They are followed by Randy Dougherty and Lynne Newman (57.35%), Steve Johnson and Mac Busby (56.62%), and Suzanne Lebending and Roger Doughman (55.75%). As noted in the La Jolla unit game payoff analysis, one can not state with certainty that any partnership is better than another but rather only state a degree of confidence that this is the case based on difference in average percentage and the error on each partnership’s average percentage. For example, Dave and Dave lead Randy and Lynne 57.55 ± 1.59% to 57.35 ± 1.30%. We must ask how confident we are that (57.55 − 57.35) = 0.20 ± 2.05% is greater than zero where the errors from each partnership have been added in quadrature. 0.20% is 0.097 standard deviations (0.20 / 2.05) away from zero. We can calculate the confidence using the error function; for example via 1 − erfc(0.097/sqrt(2)) / 2 in Matlab which yields 0.54. From this we can say with about 54% confidence that Dave and Dave are a stronger partnership than Randy and Lynne; in order words it is nearly a coin toss. Similarly we can say with about 64% confidence that Dave and Dave are stronger than Steve and Mac.

2012-2013 San Diego Unit Open Game Payoff Matrix

Both axes list the partnerships in order of decreasing strength. Each square shows how well the partnership listed on corresponding row does against the partnership in the corresponding column. The color scale runs from solid blue (30% or lower) to solid yellow (70% or better) with grey at 50%. White squares indicate partnerships that have never played any boards against each other. Pink squares indicate partnerships that can not interact because they have one or more players in common. The diagonal is always pink.

Move the mouse over the image to view the details for each square (matrix element). The hovering tooltip will show the names of the two partnerships, each partnership’s average against the field, the number of boards the partnerships have played against each other, and in bold the average results of those boards including an error. The average result shows how well the partnership on the row did against the partnership on the column. The matrix is anti-symmetric about the diagonal. Squares with a faint red X denote statistics based on relatively few boards, fewer than 10 in this case.

The upper right corner of the matrix is mostly yellow. This is because stronger partnerships usually have an advantage against weaker partnerships. Conversely the lower left corner is mostly blue.

The second matrix corrects for the difference in partnership skill, showing how much each partnership overperforms or underperforms against another partnership. The blue to yellow scale runs from -20% to 20% with grey at 0%. Move the mouse of the image to view the details for each square (matrix element). As before the partnership names and average percentage are shown. The over-perform / under-perform percentage is shown in bold. The ordinary percentage from the previous figure is also shown. Note: the previous figure also shows the over/under-performance statistics after ‘OU:’ as in ‘OU: 9.7%’.

Payoff tables and payoff graphs

The more yellowish and bluish square in the second matrix above are interesting because they indicate a pair that is strongly over/underperforming against another pair. The following table lists these relations in descending degree of payoff (or “exploitability”) where the pair-pair percentage is derived from at least 20 boards. Cnt is the number of boards that each interaction percentage based on.

Biggest Over/underperforming partnership interactions in the payoff matrix
Overperforming Pair Underperforming Pair Cnt OU%
Phyllis Pankow - Elizabeth Garber Samuel Jordan - William Grant 22 15.62
Mary Huffaker - Ronald Huffaker Suzanne LeBendig - Roger Doughman 21 12.27
Vicki Creamer - Jon Wright Phyllis Pankow - Elizabeth Garber 27 12.19
Myrel Johnson - Gregory Wetz Hanan Deeby - Bernard Figueiredo 20 11.39
Marty Bloomberg - Leila Bloomberg Roy Green - Mary Green 21 11.08
Myrel Johnson - Gregory Wetz Samuel Jordan - William Grant 20 10.98
Elaine Chan - Michael Mezin Sue Kane - Sally Ishihara 23 10.36
Barb Holles - Stephanie Rake Vicki Creamer - Jon Wright 28 9.98
Suzanne LeBendig - Roger Doughman Hanan Deeby - Bernard Figueiredo 28 9.17
Vicki Creamer - Jon Wright Ray Rowen - Alan Rowen 22 9.10
Phyllis Pankow - Elizabeth Garber John Lagodimos - Kathie Angione 36 8.71
Sue Kane - Sally Ishihara Evelyn Lantz - Fred Nimtz 20 8.25
Freda Anderson - Lynne O'Neill Manoochehr Bahmanian - Lyle Kalish 20 8.19
Lena Jelusich - Alice Lane Suzanne LeBendig - Roger Doughman 30 7.69
Andrew Loh - Patricia Loh Lena Jelusich - Alice Lane 20 7.27
John Lagodimos - Kathie Angione Barb Holles - Stephanie Rake 29 7.21
Mary Huffaker - Ronald Huffaker Phyllis Pankow - Elizabeth Garber 36 6.97
Mary Huffaker - Ronald Huffaker Lena Jelusich - Alice Lane 44 6.89
Evelyn Lantz - Fred Nimtz Myrel Johnson - Gregory Wetz 27 6.88
Roy Green - Mary Green Elaine Chan - Michael Mezin 21 6.68
John Lagodimos - Kathie Angione Dan Dayani - Bette Cornelius 20 6.58
Manoochehr Bahmanian - Lyle Kalish Lena Jelusich - Alice Lane 25 6.50
Suzanne LeBendig - Roger Doughman Phyllis Pankow - Elizabeth Garber 31 6.25
Mary Huffaker - Ronald Huffaker Freda Anderson - Lynne O'Neill 37 6.04
Barb Holles - Stephanie Rake Elaine Chan - Michael Mezin 29 5.96
Marty Bloomberg - Leila Bloomberg Sue Kane - Sally Ishihara 21 5.87
Lena Jelusich - Alice Lane John Lagodimos - Kathie Angione 35 5.85
Myrel Johnson - Gregory Wetz Mary Huffaker - Ronald Huffaker 27 5.68
Vicki Creamer - Jon Wright Myrel Johnson - Gregory Wetz 29 5.49
Ray Rowen - Alan Rowen Elaine Chan - Michael Mezin 26 5.31
Ray Rowen - Alan Rowen Evelyn Lantz - Fred Nimtz 21 5.13
David Oakley - David Walters Lena Jelusich - Alice Lane 22 5.02
Otto Newman - June Newman Myrel Johnson - Gregory Wetz 22 4.63
Hanan Deeby - Bernard Figueiredo Manoochehr Bahmanian - Lyle Kalish 20 4.59
Elaine Chan - Michael Mezin Myrel Johnson - Gregory Wetz 27 4.53
Freda Anderson - Lynne O'Neill John Lagodimos - Kathie Angione 24 4.27
Otto Newman - June Newman Lena Jelusich - Alice Lane 31 4.21
Freda Anderson - Lynne O'Neill Hanan Deeby - Bernard Figueiredo 23 4.21
Lena Jelusich - Alice Lane Evelyn Lantz - Fred Nimtz 29 4.13
Freda Anderson - Lynne O'Neill Otto Newman - June Newman 25 4.01
Mary Huffaker - Ronald Huffaker Dan Dayani - Bette Cornelius 24 3.86
Myrel Johnson - Gregory Wetz Marty Bloomberg - Leila Bloomberg 26 3.78
Ray Rowen - Alan Rowen Mary Huffaker - Ronald Huffaker 33 3.78
Elaine Chan - Michael Mezin Lena Jelusich - Alice Lane 40 3.59
Barb Holles - Stephanie Rake Mary Huffaker - Ronald Huffaker 50 3.53
Hanan Deeby - Bernard Figueiredo Samuel Jordan - William Grant 23 3.31
Barb Holles - Stephanie Rake Marty Bloomberg - Leila Bloomberg 32 3.19
Manoochehr Bahmanian - Lyle Kalish Myrel Johnson - Gregory Wetz 24 3.03

The table gives us much to speculate about and yet it is still a lot to absorb. Part of the difficulty is that many pairs are listed multiple times. A graph may be a better representation. Here are two examples: OU ≥ 5% and OU ≥ 3%. The first graph represents the data in the table above down to advantages as small as 5% and the second represents the full table.

Unlike the Social Network graphs, the payoff graphs are directed, i.e they represent flow, the transport of over/underperformance around the partnership network. An incoming arrow represents a payoff to a partnership; an outgoing arrow represents payoff from a partnership.

It should be noted that although these are directed graphs, they are not directed acyclic graphs (DAGs), an important and well studied class of graphs. Cycles are easy to spot. For example Pankow-Garber pays off to Creamer-Wright who pay off to Holles-Rake, who payoff to Angione-Lagodimos, who complete the cycle by paying off to Pankow-Garber. So turns the wheel of exploitation.

Scientists and mathematicians may be concerned about the fact that inflow and outflow are not equal at each node (pair) given that the over/underperformance numbers have compensated for the difference in partnership ability. There are two reasons for this. First, only the most prominent payoffs are shown. Many smaller payoffs are suppressed in order to avoid total clutter. Second, the payoff calculations are based on different numbers of boards. For example, an underperformance of 3% over 40 boards would be balanced by a overperformance of 6% over 20 boards.

2012-2013 Combined La Jolla and San Diego Unit Game Payoff Matrix

There is considerable overlap between the players in the La Jolla unit and San Diego unit open games because one unit has its games on the first and third Sundays and the other has theirs on the second and forth Sundays of every month. It makes sense to combine the two data sets for better statistics. For this analysis we will only consider pairs that have played at least 20 session between the two games over the last two years.

The leaders are Randall Dougherty and Lynne Newman (57.49 ± 0.99%), David Oakley and Dave Walters (57.23 ± 1.10%), Steve Johnson and Mac Busby (55.96 ± 1.18%), Suzanne Lebendig and Roger Doughman (55.74 ± 0.67%), and Lynne O’Neill and William Grant (55.27 ± 0.75%). The more than 1% gap between the second and third place partnerships is quite significant. It means we can say with 78% confidence that Dave and Dave are better than Mac and Steve, up from the roughly 65% confidence based on either unit’s game results.

Here is the over/underperformance matrix.

Payoff tables and payoff graphs

The following table lists these relations in descending degree of payoff (or “exploitability”) where the pair-pair percentage is derived from at least 20 boards. Cnt is the number of boards that each interaction percentage based on.

Biggest Over/underperforming partnership interactions in the payoff matrix
Overperforming Pair Underperforming Pair Cnt OU%
Andrew Loh - Patricia Loh Mary Huffaker - Ronald Huffaker 20 17.84
Phyllis Pankow - Elizabeth Garber Samuel Jordan - William Grant 22 15.51
Freda Anderson - Gail Dunham Alice Lane - Dorinda Lindvall 22 14.36
Alice Lane - Dorinda Lindvall Marty Bloomberg - Leila Bloomberg 21 13.61
Kent Hartman - Maritha Pottenger Roy Green - Mary Green 23 13.22
Vicki Creamer - Jon Wright Phyllis Pankow - Elizabeth Garber 27 12.37
John Peter Lagodimos - Kathie Angione Evelyn Lantz - Charlotte Blum 21 11.82
Myrel Johnson - Gregory Wetz Hanan Deeby - Bernard Figueiredo 20 11.53
Elaine Chan - Michael Mezin Sue Kane - Sally Ishihara 28 11.16
Myrel Johnson - Gregory Wetz Samuel Jordan - William Grant 20 10.98
Barb Holles - Stephanie Rake Vicki Creamer - Jon Wright 28 9.90
Phyllis Pankow - Elizabeth Garber John Peter Lagodimos - Kathie Angione 36 9.57
Marty Bloomberg - Leila Bloomberg Lynne Anderson - Ursula Kantor 26 9.44
Suzanne Lebendig - Roger Doughman Marty Bloomberg - Leila Bloomberg 20 9.26
Marty Bloomberg - Leila Bloomberg Carolyn Casey - Lena Jelusich 26 9.11
Lynne O'Neill - William Grant Elaine Chan - Michael Mezin 20 8.92
Lynne Anderson - Ursula Kantor Freda Anderson - Gail Dunham 29 8.39
Lena Jelusich - Alice Lane Suzanne Lebendig - Roger Doughman 34 8.36
Charles Wilson Jr - Barbara Norman Matthew Kidd - Patricia Lane 20 8.32
Ray Rowen - Alan Rowen Freda Anderson - Gail Dunham 20 8.24
Mary Huffaker - Ronald Huffaker Lynne Anderson - Ursula Kantor 42 8.17
Kent Hartman - Maritha Pottenger Dan Dayani - Bette Cornelius 30 8.06
Andrew Loh - Patricia Loh Lena Jelusich - Alice Lane 22 7.94
Freda Anderson - Lynne O'Neill Manoochehr Bahmanian - Lyle Kalish 20 7.85
Kent Hartman - Maritha Pottenger Ray Rowen - Alan Rowen 34 7.84
Phyllis Pankow - Betty-Jo Petersen Carolyn Casey - Lena Jelusich 20 7.67
Evelyn Lantz - Fred Nimtz Myrel Johnson - Gregory Wetz 27 7.02
Barb Holles - Stephanie Rake Elaine Chan - Michael Mezin 29 7.01
Manoochehr Bahmanian - Lyle Kalish Lena Jelusich - Alice Lane 25 7.00
Myrel Johnson - Gregory Wetz Mary Huffaker - Ronald Huffaker 27 6.95
Suzanne Lebendig - Roger Doughman Charles Wilson Jr - Barbara Norman 22 6.94
Phyllis Pankow - Betty-Jo Petersen Alice Lane - Dorinda Lindvall 21 6.89
Dan Dayani - Bette Cornelius Marty Bloomberg - Leila Bloomberg 34 6.88
Freda Anderson - Gail Dunham John Peter Lagodimos - Kathie Angione 22 6.81
Charles Wilson Jr - Barbara Norman Evelyn Lantz - Charlotte Blum 29 6.74
Charles Wilson Jr - Barbara Norman Andrew Loh - Patricia Loh 21 6.74
Lena Jelusich - Alice Lane John Peter Lagodimos - Kathie Angione 35 6.65
Lynne Anderson - Ursula Kantor Elaine Chan - Michael Mezin 27 6.62
Evelyn Lantz - Charlotte Blum Lynne Anderson - Ursula Kantor 23 6.53
John Peter Lagodimos - Kathie Angione Dan Dayani - Bette Cornelius 27 6.43
Mary Huffaker - Ronald Huffaker Lena Jelusich - Alice Lane 48 6.42
Suzanne Lebendig - Roger Doughman Phyllis Pankow - Elizabeth Garber 31 6.37
Sue Kane - Sally Ishihara Evelyn Lantz - Fred Nimtz 23 6.35
John Peter Lagodimos - Kathie Angione Barb Holles - Stephanie Rake 29 6.24
Phyllis Pankow - Betty-Jo Petersen Ray Rowen - Alan Rowen 20 6.23
John Peter Lagodimos - Kathie Angione Otto Newman - June Newman 31 6.20
Otto Newman - June Newman Lynne O'Neill - William Grant 23 5.93
Ray Rowen - Alan Rowen Andrew Loh - Patricia Loh 26 5.84
Carolyn Casey - Lena Jelusich Suzanne Lebendig - Roger Doughman 22 5.82
Mary Huffaker - Ronald Huffaker Phyllis Pankow - Elizabeth Garber 40 5.79
Suzanne Lebendig - Roger Doughman Kent Hartman - Maritha Pottenger 42 5.71
Vicki Creamer - Jon Wright Myrel Johnson - Gregory Wetz 29 5.56
Phyllis Pankow - Betty-Jo Petersen Lynne O'Neill - William Grant 31 5.52
David Oakley - David Walters Lena Jelusich - Alice Lane 22 5.51
Dan Dayani - Bette Cornelius Elaine Chan - Michael Mezin 45 5.42
Mary Huffaker - Ronald Huffaker Lynne O'Neill - William Grant 27 5.39
Otto Newman - June Newman Sue Kane - Sally Ishihara 21 5.29
Freda Anderson - Lynne O'Neill John Peter Lagodimos - Kathie Angione 24 5.24
Roy Green - Mary Green Otto Newman - June Newman 35 5.22
Samuel Jordan - Yoko Davis Alice Lane - Dorinda Lindvall 24 5.20
Dan Dayani - Bette Cornelius Suzanne Lebendig - Roger Doughman 28 5.17
Mary Huffaker - Ronald Huffaker Suzanne Lebendig - Roger Doughman 35 5.12

Here are corresponding graphs for OU ≥ 7% and OU ≥ 5%. The first graph represents the data in the table above down to advantages as small as 7% and the second represents the full table. Observe that the first graph is composed of three independent graphs, one as simple as the payoff from Kidd-Lane to Wilson-Norman.

Get the data

Download a zip file (3 MB) of all the tab delimited text files created by the Payoff Matrix software and used to generate the payoff matrix images and graphs above. For convenience, an Excel version of each file is also included. A filename such as U539-2012-2013-min-10-payoff-matrix.txt means the payoff matrix data for the 2012-2013 time period after restricting to partnerships that have played at least 10 sessions. If min-# is not included in the filename, no minimum session cut has been applied. The designation U526+U539 indicates data from combining the two unit games. The definition of each column is explained in the Payoff Matrix software documentation. The zip file also includes the images, the image specific JavaScript files, the Graphviz (.gvz) files, and the Scalable Vector Graphics files (.svg) for the graphs. A filename such as U539-2012-2013-mb-20-lp-3.gvz means results based on at least 20 boards (mb = “minimum boards”) and at least a 3% advantage (lp = “lowest percentage”).