create observations

Reviewed-on: https://git.infra.nkode.tech/dkelly/evilnkode/pulls/5

.gitignore

@@ -2,3 +2,5 @@
 .DS_Store
 output
 __pycache__
+.ipynb_checkpoints
+

docs/nkode_analysis.md

@@ -1,19 +1,20 @@
- # nKode analysis
+# nKode analysis
 
 Luke Oeding (Auburn University)
 
 ## What is an nKode?
+
 An nKode consists of a passcode $P$ selected utilizing the following setup:
 
-* a $k$ digit keypad, where each key has:
+- a $k$ digit keypad, where each key has:
-* $p$ properties (position, central number, color, letter, emoji,...)
+- $p$ properties (position, central number, color, letter, emoji,...)
-* $m$ options for each property (\# positions, \# central numbers, \# colors, \# letters, \# emoji, ... )
+- $m$ options for each property ($\#$ positions, $\#$ central numbers, $\#$ colors, $\#$ letters, $\#$ emoji, ... )
 
   - In principle the number of options for each property doesn't have to be the same, but for the sake of our analysis, we make this uniform choice.
   - Typically one wants to have all letters displayed exactly once on the keypad for any instance of a keypad, so we take $m = k$.
 
-* $\ell$ = length of the passcode, which is a sequence of $\ell$ letters (options) selected from any of the options. 
+- $\ell$ = length of the passcode, which is a sequence of $\ell$ letters (options) selected from any of the options.
-* a shuffling rule (the split-shuffle)
+- a shuffling rule (the split-shuffle)
 
 ## Split shuffling
 
@@ -46,14 +47,14 @@ $$ E(m,p,\ell) = \log_2[(mp)^\ell] = \ell \frac{\log(mp)}{\log(2)}$$
 
 So, for example, when $m=10$ and $p = 7$, the alphabet has 70 letters, and a passcode of length $\ell = 4$ would have entropy:
 
-$$ E(10,7,4) =  4\log(70)/\log(2) = 24.5\; \text{bits}$$
+$$E(10,7,4) = 4\log(70)/\log(2) = 24.5\; \text{bits}$$
 
-$$ E(10,7,6) =  6\log(70)/\log(2) = 36.8\; \text{bits}$$
+$$E(10,7,6) = 6\log(70)/\log(2) = 36.8\; \text{bits}$$
 
-Relevant is the entropy per symbol, which is $ E(m,p,1)  = \log_2(m) +\log_2(p) = \frac{\log(m) + \log(p)}{\log(2)}$.
+Relevant is the entropy per symbol, which is $E(m,p,1) = \log_2(m) +\log_2(p) = \frac{\log(m) + \log(p)}{\log(2)}$.
 
 For instance:
-$ E(10,7,1) =  \log(70)/\log(2) = 6.13\; \text{bits}$
+$E(10,7,1) = \log(70)/\log(2) = 6.13\; \text{bits}$
 
 ### nKode Entropy
 
@@ -62,16 +63,16 @@ If one is interested in the likelihood of a single attempt randomly entered nKod
 The total number of available passcodes with
 $\ell$ letters selected from an alphabet with $k$ keys, with replacement is
 
-$$ k^\ell $$
+$$k^\ell$$
 
 So the entropy of the one time keypad is
 
-$$ E(k,\ell) = \log_2[(k)^\ell] = \ell \frac{\log(k)}{\log(2)}$$
+$$E(k,\ell) = \log_2[(k)^\ell] = \ell \frac{\log(k)}{\log(2)}$$
 
 So, for example, when $k=10$ a passcode of length $\ell = 4$ would have entropy:
-$ E(10,4) =  4\log(10)/\log(2) = 13.28\; \text{bits}$
+$E(10,4) = 4\log(10)/\log(2) = 13.28\; \text{bits}$
 
-The entropy per symbol, which is $ E(k,1)  = \log_2(k) = \frac{\log(k)}{\log(2)}$.
+The entropy per symbol, which is $E(k,1) = \log_2(k) = \frac{\log(k)}{\log(2)}$.
 
 In the case $k=10$ we have $E(10,1) = 3.22 \; \text{bits}$.
 
@@ -83,7 +84,6 @@ We are interested in understanding the number of times an eavesdropping intruder
 
 [Brooks Brown says that the lower bound is $\min\{3, s \log_2+1\}$, where $s$ is the number of attribute sets, for an nKode of complexity $c=1$, regardless of nKode length $l$, or the number of tiles $t$.]
 
-
 Regarding the eavesdropper attack we should also consider the case of a keystroke recorder that doesn't observe the labels on the keys of the nKode keypad.
 Blind single attacks and repeated blind attempts that successfully log in, and learning the nKode.
 
@@ -95,16 +95,11 @@ Blind single attacks and repeated blind attempts that successfully log in, and l
 
 The probability that a (blind) randomly entered key-string will yield a successful login:
 
-$
+$R = \frac{\#(\text{passcodes that would yield a correct login})}{\#(\text{possible passcodes})}.$
-R = \frac{\text{Num}(\text{passcodes that would yield a correct login})}{\text{Num}(\text{possible passcodes})}.
-$
 
-(Num stands for 'number of'.)
 This should be simply computed using the number of keys $k$ and the length $l$ of the passcode:
 
-$
+$R = (1/k)^l.$
-R = (1/k)^l.
-$
 
 For example, a 6 digit pin would yield a one-in-a-million chance of blindly hitting the correct passcode.
 
@@ -114,7 +109,7 @@ For the moment we only consider the case of $p=2$, which could be the case if th
 
 Note that if a user insists on only using the placement value of the keys in an nKode to select their password, then they effectively have reduced the complexity or entropy of their password to that of the normal keypad, and an adversary could use frequency analysis to increase the likelihood of guessing a correct password. However, if the user were to use only the number values on the keypad, they would still only have the entropy of a standard keypad generated password, however, because of the shuffling of the letters, using an nKode provides the user with some protection against frequency analysis attacks in the case of a blind intruder.
 
-For example, it is know that users pick passwords like 1234 or 123456 much more frequently than of any other password. If the intruder is able to observe the user typing one of these passcodes on an nKode keypad, then they would be able to guess the passcode with higher probability than random. However, if the intruder is only able to record keystrokes, but not see the display of the nKode keypad, then the intruder would only guess the correct passcode at the frequency of guessing a permutation of the correct passcode. In the case of $\ell $ consecutive digits, the keystroke intruder would only observe $\ell$ distinct keys being typed. The number of such passcodes is $\binom{k}{\ell}$. So in the case of a $k=10$ digit keypad, the number of $\ell=4$ digit passcodes with distinct entries is $\binom{10}{4} = 210$, and when $\ell = 6$ we also have $\binom{10}{6} = 210$. So, after one observation indicating that the passcode consists of $\ell$ distinct digits, the intruder would have probability $P = 1/\binom{k}{\ell}$ of guessing the passcode. 
+For example, it is know that users pick passwords like 1234 or 123456 much more frequently than of any other password. If the intruder is able to observe the user typing one of these passcodes on an nKode keypad, then they would be able to guess the passcode with higher probability than random. However, if the intruder is only able to record keystrokes, but not see the display of the nKode keypad, then the intruder would only guess the correct passcode at the frequency of guessing a permutation of the correct passcode. In the case of $\ell$ consecutive digits, the keystroke intruder would only observe $\ell$ distinct keys being typed. The number of such passcodes is $\binom{k}{\ell}$. So in the case of a $k=10$ digit keypad, the number of $\ell=4$ digit passcodes with distinct entries is $\binom{10}{4} = 210$, and when $\ell = 6$ we also have $\binom{10}{6} = 210$. So, after one observation indicating that the passcode consists of $\ell$ distinct digits, the intruder would have probability $P = 1/\binom{k}{\ell}$ of guessing the passcode.
 It is known that the expected number of trials until the first success would be $1/P$. In this case it would take the intruder on average $\binom{k}{\ell}$ attempts to successfully log in (without guessing the passcode). Even in the case of the 4 digit PIN where the intruder guesses that the passcode consists of the first $4$ digits, (or consecutive or even just distinct) integers the nKode obtains an increase in security for the user by a factor of approximately 210 since the key recording intruder would guess the password 1234 on the first attempt on a standard keypad, but would need approximately 210 trials to successfully log in.
 
 ### Multiple blind attempts for a large userbase (Password spraying)
@@ -143,13 +138,9 @@ For example,
 ### Guessing the passcode:
 
 The likelihood that a randomly chosen passcode will yield a successful login no matter what shuffle has been applied, i.e. so that the attacker can successfully log in as many times as they want:
-$1 / \text{Num}(\text{possible passcodes}).$
+$1 / \#(\text{possible passcodes}).$
 
-For example, when $k=m=10, p=7$ for $\ell = 4$ this probability is 
+For example, when $k=m=10, p=7$ for $\ell = 4$ this probability is $(70)^{-4} \sim 4.16*10^{-8}$, or about 4 chances in 100 million, and for $\ell = 6$ this probability is $(70)^{-6} \sim 8.5*10^{-12}$, or about 8.5 chances in 1 trillion.
-
-$(70)^{-4} \sim 4.16*10^{-8},$
-
- or about 4 chances in 100 million, and for $\ell = 6$ this probability is $(70)^{-6} \sim 8.5*10^{-12}$, or about 8.5 chances in 1 trillion.  
 
 ## Multiple Blind Attempts
 
@@ -164,6 +155,7 @@ For example, when $k=10$ for $\ell = 4$ and $s=2$ this probability is $((10)^{-4
 For example, when $k=10$ for $\ell = 4$ and $s=3$ this probability is $((10)^{-4})^3 = 10^{-12}$, [same order of magnitude as a passcode of length 6], and for $\ell = 6$ this probability is $((10)^{-6})^3 = 10^{-18}$, or 1 chance in one billion billion.
 
 ## Incorrect nKodes that still work
+
 We should also consider the number of nKodes that would yield a successful sequence of $s$ logins. [Add example]
 
 ## Longer passcodes might not always be more secure
@@ -193,348 +185,3 @@ If someone is able to observe the user typing the nKode, or record the keystroke
 ### Eye tracking
 
 Eye tracker on phone: How good would this need to be in order to see what attribute the user is searching for?
-
-# Higher Complexity
-
-## Dispersion
-Dispersion is an operation on a keypad that permutes properties in such a way that 2 observations of the nKode are sufficient to learn the passcode. It does this by applying a distinct rotation to each property. The authors note that this is possible when the number of keys is not larger than the number of properties per key, because this ensures that there are enough distinct rotations so that no repetitions occur.  There are more general permutations that can also have this property, and it seems that this is already implemented in the Enrollment_Login_Renewal. 
-
-## Split shuffle
-Split shuffle attempts to avoid the dispersion permutation of the keypad so as to increase the number of times an intruder would have to observe the nKode being entered.
-
-The properties are divided into 2 sets, each set will be shuffled by the same shuffle applied to all properties in that set of properties.
-
-Note: by observing both keypads (before and after a split-shuffle), one can learn both what the split was, and what the two shuffles were.
-
-We're intertested in studying how many observations an intruder must make in order to learn the passcode with the split shuffle in place. There are a few scenarios I can imagine. 
-
-    * No split (analyzed above).
-    * The split is determined once, and then the shuffles only happen on one side of the split.
-    * The split changes every time randomly.
-    * The split changes every time by a set strategy.
-
-Of course we can consider these questions each time the metaparameters change. Recall,  
-    * a $k$ digit keypad,
-    * $p$ properties (position, central number, color, letter, emoji,...)
-    * $m$ options for each property (\# positions, \# central numbers, \# colors, \# letters, \# emoji, ... ). typically  $m = k$.
-    * $\ell$ = length of the passcode, which is a sequence of $\ell$ letters (options) selected from any of the options. 
-
-Notice that when $k = 1$ the problem is nearly trivial. It doesn't matter what the properties are, the only thing the user is entering is $\ell$, and that can be observed in 1 try.
-
-When $k = 2$. Here's the case $p = 2$ and a 4 letter passcode.
-
-      
-|key   | p0 | p1 |
-|:----:|:--:|:--:|
-|key 0 | a0 | b0 |
-|key 1 | a1 | b1 |
-
-After a shuffle [attribute 1]
-
-|key   | p0 | p1 |
-|:----:|:--:|:--:|
-|key 0 | a0 | b1 |
-|key 1 | a1 | b0 |
-
-interaction:
-
-|what |  |||||
-|--------|--|--|--|--|--|
-|Passcode| a0|b1|a1|b0|
-|Display 1| 0|1|1|0|
-|Display 2| 0|0|1|1|
-
-The possible passcodes after display 1: 
-0{a0,b0},1{a1,b1},1{a1,b1},0{a0,b0}
-
-The possible passcodes after display 2: 
-0{a0,b1},0{a0,b1},1{a1,b0},1{a1,b0}
-
-Intersect:
-{a0},{b1},{a1},{b0}
-Passcode learned in 2.
-
-
-When $k = 2$. Here's the case $p = 4$ and a 4 letter passcode.
-
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b0  | c0  | d0  |
-|key 1 | a1  | b1  | c1  | d1  |
-
-After a shuffle [attribute 1,2]
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b1  | c1  | d0  |
-|key 1 | a1  | b0  | c0  | d1  |
-
-
-interaction:
-
-|what |  |||||
-|--------|--|--|--|--|--|
-|Passcode| a0|c1|c1|d0|
-|Display 1| 0|1|1|0|
-|Display 2| 0|0|0|0|
-
-The possible passcodes after display 1: 
-0{a0,b0,c0,d0},1{a1,b1,c1,d1},1{a1,b1,c1,d1},0{a0,b0,c0,d0}
-
-The possible passcodes after display 2: 
-0{a0,b1,c1,d0},0{a0,b1,c1,d0},0{a0,b1,c1,d0},0{a0,b1,c1,d0}
-
-Intersect:
-{a0,d0},{b1,c1},{b1,c1},{a0,d0}.
-
-Passcode is not learned yet.
-
-After a 2nd shuffle [attribute 1,3]
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b1  | c0  | d1  |
-|key 1 | a1  | b0  | c1  | d0  |
-
-|what |  |||||
-|--------|--|--|--|--|--|
-|Passcode| a0|c1|c1|d0|
-|Display 1| 0|1|1|0|
-|Display 2| 0|0|0|0|
-|Display 3| 0|1|1|1|
-
-The possible passcodes after display 3: 
-0{a0,b1,c0,d1},1{a1,b0,c1,d0},1{a1,b0,c1,d0},1{a1,b0,c1,d0}
-
-Intersect:
-{a0},{c1},{c1},{d0}.
-
-Passcode learned in 3.
-
-
-Here's the case $p = 4$ and an 8 letter passcode.
-
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b0  | c0  | d0  |
-|key 1 | a1  | b1  | c1  | d1  |
-
-Shuffle 1 [attribute 1,2]
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b1  | c1  | d0  |
-|key 1 | a1  | b0  | c0  | d1  |
-
-Shuffle 2 [attribute 1,3]
-
-|key   | p0 | p1 | p2 | p3 |
-|:----:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b1  | c0  | d1  |
-|key 1 | a1  | b0  | c1  | d0  |
-
-interaction:
-
-|what |  ||||||||||
-|--------|--|--|--|--|--|--|--|--|--|--|
-|Passcode| a0|c1|c1|d0|b1|b0|a1|d0|
-|Display 1| 0| 1| 1| 0| 1| 0| 1| 0|
-|Display 2| 0| 0| 0| 0| 0| 1| 1| 0|
-|Display 3| 0| 1| 1| 1| 0| 1| 1| 1|
-
-The possible passcodes after display 1: 
-0{a0,b0,c0,d0},1{a1,b1,c1,d1},1{a1,b1,c1,d1},0{a0,b0,c0,d0},
-1{a1,b1,c1,d1},0{a0,b0,c0,d0},1{a1,b1,c1,d1},0{a0,b0,c0,d0}
-
-The possible passcodes after display 2: 
-0{a0,b1,c1,d0},0{a0,b1,c1,d0},0{a0,b1,c1,d0},0{a0,b1,c1,d0},
-0{a0,b1,c1,d0},1{a1,b0,c0,d1},1{a1,b0,c0,d1},0{a0,b1,c1,d0}
-
-Intersect:
-{a0,d0},{b1,c1},{b1,c1},{a0,d0},{b1,c1},{b0,c0},{a1,d1},{a0,d0}
-
-Passcode is not learned yet.
-
-The possible passcodes after display 3: 
-0{a0,b1,c0,d1},1{a1,b0,c1,d0},1{a1,b0,c1,d0},1{a1,b0,c1,d0},
-0{a0,b1,c0,d1},1{a1,b0,c1,d0},1{a1,b0,c1,d0},1{a1,b0,c1,d0}
-
-Intersect:
-{a0},{c1},{c1},{d0},{b1},{b0},{a1},{d0}
-
-Passcode learned in 3.
-
-
-Here's the case $p = 8$ and an 4 letter passcode.
-
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b0  | c0  | d0  | e0  | f0  | g0  | h0  |
-|key 1 | a1  | b1  | c1  | d1  | e1  | f1  | g1  | h1  |
-
-Shuffle 1 [attribute 0,2,4,6] [a,c,e,g]
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a1  | b0  | c1  | d0  | e1  | f0  | g1  | h0  |
-|key 1 | a0  | b1  | c0  | d1  | e0  | f1  | g0  | h1  |
-
-Shuffle 2 [attribute 2,3,4,5] [c,d,e,f]
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b0  | c1  | d1  | e1  | f1  | g0  | h0  |
-|key 1 | a1  | b1  | c0  | d0  | e0  | f0  | g1  | h1  |
-
-Shuffle 3 [attribute 0,4,5,6] [a,e,f,g]
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a1  | b0  | c0  | d0  | e1  | f1  | g1  | h0  |
-|key 1 | a0  | b1  | c1  | d1  | e0  | f0  | g0  | h1  |
-
-
-interaction:
-
-|what |  ||||||||||
-|--------|--|--|--|--|--|--|--|--|--|--|
-|Passcode| a1|c1|b0|h0|f0|e1|a1|d0|
-|Display 1| 1| 1| 0| 0| 0| 1| 1| 0|
-|Display 2| 0| 0| 0| 0| 0| 0| 0| 0|
-|Display 3| 1| 0| 0| 0| 1| 0| 1| 1|
-|Display 4| 0| 1| 0| 0| 1| 0| 0| 0|
-
-The possible passcodes from display 1:  
-`1{a1,b1,c1,d1,e1,f1,g1,h1},`  
-`1{a1,b1,c1,d1,e1,f1,g1,h1},`  
-`0{a0,b0,c0,d0,e0,f0,g0,h0},`  
-`0{a0,b0,c0,d0,e0,f0,g0,h0},`  
-`0{a0,b0,c0,d0,e0,f0,g0,h0},`  
-`1{a1,b1,c1,d1,e1,f1,g1,h1},`  
-`1{a1,b1,c1,d1,e1,f1,g1,h1},`  
-`0{a0,b0,c0,d0,e0,f0,g0,h0}`
-
-The possible passcodes from display 2:  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0},`  
-`0{a1,b0,c1,d0,e1,f0,g1,h0}`
-
-Intersect:  
-`1{a1,c1,e1,g1},`  
-`1{a1,c1,e1,g1},`  
-`0{b0,d0,f0,h0},`  
-`0{b0,d0,f0,h0},`  
-`0{b0,d0,f0,h0},`  
-`1{a1,c1,e1,g1},`  
-`1{a1,c1,e1,g1},`  
-`0{b0,d0,f0,h0}`  
-
-Passcode is not learned yet.
-
-Shuffle 2 [attribute 2,3,4,5] [c,d,e,f]
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a0  | b0  | c1  | d1  | e1  | f1  | g0  | h0  |
-|key 1 | a1  | b1  | c0  | d0  | e0  | f0  | g1  | h1  |
-
-So before observing the input for the 3rd attempt:
-Hits for each key from prior information:
-[1001][1001]
-(1/2 are 1's are 1/2 are 0's for every key.)
-
-Now observe the sequence 10001011
-The possible passcodes from display 3:  
-`1{a1,b1,c0,d0,e0,f0,g1,h1},`  
-`0{a0,b0,c1,d1,e1,f1,g0,h0},`  
-`0{a0,b0,c1,d1,e1,f1,g0,h0},`  
-`0{a0,b0,c1,d1,e1,f1,g0,h0},`  
-`1{a1,b1,c0,d0,e0,f0,g1,h1},`  
-`0{a0,b0,c1,d1,e1,f1,g0,h0},`  
-`1{a1,b1,c0,d0,e0,f0,g1,h1},`  
-`1{a1,b1,c0,d0,e0,f0,g1,h1}`  
-
-Intersect with prior information:  
-`1{a1,g1},`  
-`0{c1,e1},`  
-`0{b0,h0},`  
-`0{b0,h0},`  
-`1{d0,f0},`  
-`0{c1,e1},`  
-`1{a1,g1},`  
-`1{d0,f0}`  
-
-Passcode not learned yet, but after one more shuffle, we think we would learn the passcode.
-
-Shuffle 3 [attribute 0,4,5,6] [a,e,f,g]
-
-|key   | p0 | p1 | p2 | p3 | p4 | p5 | p6 | p7 |
-|:----:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
-|key 0 | a1  | b0  | c0  | d0  | e1  | f1  | g1  | h0  |
-|key 1 | a0  | b1  | c1  | d1  | e0  | f0  | g0  | h1  |
-
-So before observing the input for the 4th attempt:  
-Hits for each key from prior information:  
-[0][10][0][0][0][10][0][01]  
-(So several of the correct keys are identified, only letters in positions 1,5,7 are unknown)
-
-
-The possible passcodes from display 4:   
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-`1{a0,b1,c1,d1,e0,f0,g0,h1},`  
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-`1{a0,b1,c1,d1,e0,f0,g0,h1},`  
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-`0{a1,b0,c0,d0,e1,f1,g1,h0},`  
-
-Intersect:  
-`{a1,g1},`  
-`{c1},`  
-`{b0,h0},`  
-`{b0,h0},`  
-`{f0},`  
-`{e1},`  
-`{a1,g1},`  
-`{d0}`  
-
-Passcode only paritally learned after 3 shuffles
-- uncertain about positions 0,2,3,6  = {a1 or g1}, {b0 or h0}
-- certain about positions 1,4,5,7 = c1,f0,e1,d0
-
-Shuffles:  
-[],   
-[attribute 0,2,4,6] [a c e g ]  
-[attribute 2,3,4,5] [  cdef  ]  
-[attribute 0,4,5,6] [a   efg ]  
-1,7 [b,h] - not shuffled 4 times, so the key pressed was always the same for the 3rd and 4th character.
-
-3 -- shuffled 1 time, not shuffled 3 times  
-4 -- shuffled 3 times, not shuffled 1 time  
-2,5,6 -- shuffled 2 times, not shuffled 2 times  
-
-Note that this passcode didn't use the letter g, 
-
-This experiment makes us think of two learning tasks. 
-1) The ability to learn any possible passcode character from observations of inputs
-2) The liklihood of learning a given passcode of a fixed length.
-3) Could you get the correct entry even with partial information?
-4) Probabilistic attack for guessing the correct sequence? 
-    First split shuffle evenly splits things, so you don't gain much. Subsequent shuffles will start to bias toward the correct passcode. The trade-off is that if you don't shuffle a position then the intruder can just use the same input as before and can enter the correct key without knowing the correct icon, but if you do shuffle, then the intruder learns more information.
-
-
-
-One key I’m understanding better now: There’s a trade-off between (1) information being learned by the intruder and (2) the chances that an intruder could key in a correct key-sequence for each subsequent observation / login attempt.
-
-If an attribute is not shuffled, then that increases the chance for (2), and if an attribute is shuffled that increases the chance for (1). 
-
-I think that when the selection of the sets is randomized like you’re doing, you’re getting the optimal trade-off between these two things. However, there’s also probably a strategy that might be even more optimal than just randomly choosing a split each time. If each shuffle chooses for its set of half of the attributes: half from the attributes that were previously shuffled and half from the attributes that weren’t shuffled in the previous shuffle. You could look back essentially log_2(p) steps (where p is the number of attributes) and similarly subdivide. This seems to be a way to balance the tradeoff for consecutive observations.