Reaction to CUPED article

On September 15, 2024, Craig Sexauer posted “CUPED Explained” where CUPED stands for Controlled-experiment Using Pre-Experiment Data, billed as “one of the most powerful algorithmic tools for increasing the speed and accuracy of experimentation programs.”

Here is a list of some of the things that just make me want to vomit about this explanation of the tool (if not the actual tool itself):

As an experiment matures and hits its target date for readout, it’s not uncommon to see a result that seems to be only barely outside the range where it would be treated as statistically significant. 

Only barely“. As in, we already gave you a 1-in-20 chance of not proving your hypothesis, you just need a little bit more. This is very reminiscent of this XKCD comic showing that green jelly beans linked to acne, where “only 5% chance of coincidence” prominently displayed, while burying the other Red, Turquoise, Magenta, Yellow, Grey, etc (19 other times) studies. Any study that admits to using CUPED would instantly get a “what are you not telling me?” rejection.

Waiting for more samples delays your ability to make an informed decision, and it doesn’t guarantee you’ll observe a statistically significant result when there is a real effect.

See above – you have a weak hypothesis, and adding more sample doesn’t guarantee the result you want. This is all true – it just hides the “use CUPED to solve this problem” conclusion.

Conceptually, if one group has a faster average baseline, their experiment results will also be faster. When we apply a CUPED correction, the faster group’s metric will be adjusted downwards relative to the slower group.

Before you just pick a random variable and “correct” for it (like, by using CUPED), I would recommend you read (and understand) The Book of Why by Judea Pearl. This book has tons of examples where a “confounding variable” is corrected for incorrectly. The book describes p-calculus as a way to model causality [All of this is not to say I understand p-calculus. I only understand enough to tell when other people are wrong.]

Python libraries on causaility:

Posted in Machine Learning | Comments Off on Reaction to CUPED article

VT-D Very Annoying

I really love my HP Z820 Workstation. Two Xeon CPUs, 256GB of RAM. Runs VMWare. The fans scream like a banshee when it is powered up.

However, when the power goes out (happens three+ times a year), it has the habit of “forgetting” that VT-X and VT-d are enabled. So, I have to go into the BIOS, and find the setting to turn it back on again.

Just documenting (because “Security” is never where I expect to find this setting):

Posted in Computer Builds, Hardware | Comments Off on VT-D Very Annoying

Fix for GraphQL testing

You are here because you are following the “Building a GraphQL service“, and specifically the testing section using GraphQlTest, and you keep getting a runtime error:

java.lang.IllegalStateException: Failed to load ApplicationContext for [MergedContextConfiguration@5b202ff testClass = com.tiemens.fullstack.controller.BookControllerTests, locations = [], classes = [com.tiemens.fullstack.FullstackApplication], contextInitializerClasses = [], activeProfiles = [], propertySourceDescriptors = [], propertySourceProperties = ["org.springframework.boot.test.autoconfigure.graphql.GraphQlTestContextBootstrapper=true"]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphQlSource' defined in class path resource [org/springframework/boot/autoconfigure/graphql/GraphQlAutoConfiguration.class]: Failed to instantiate [org.springframework.graphql.execution.GraphQlSource]: Factory method 'graphQlSource' threw exception with message: Name for argument of type [java.lang.String] not specified, and parameter name information not found in class file either.
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.graphql.execution.GraphQlSource]: Factory method 'graphQlSource' threw exception with message: Name for argument of type [java.lang.String] not specified, and parameter name information not found in class file either.
Caused by: java.lang.IllegalArgumentException: Name for argument of type [java.lang.String] not specified, and parameter name information not found in class file either.

The fix is in BookController, bookById: simply change “@Argument String id” to be “@Argument(“id”) String id”, as shown below:

@Controller
public class BookController {
    @QueryMapping
    public Book bookById(@Argument("id") String id) {
        return Book.getById(id);
    }
    ...etc...
}

Magically injected error that is magically fixed.

Note: this was caused by Spring Boot v3,4,1 (or 3.4, or 3.2 or something in the “3”s)

Posted in Software Engineering | Comments Off on Fix for GraphQL testing

Fix for ‘remote-git’ is not a git command

Just documenting another weird error message.

If you see

git: 'remote-git' is not a git command. See 'git --help'.

Then, in your .git/config file, change this line:

	url = git::/reposerver/wordsearch

to this:

	url = git://reposerver/wordsearch

and try your git command again.

Posted in Software Project | Comments Off on Fix for ‘remote-git’ is not a git command

SSH path canonicalization failed

Yet another “change stuff just to change stuff” explanation and fix.

Problem: you see “No such file” or “path canonicalization failed” after an scp command line that used to work correctly.

tim@timub2404:~/eclipse-work$ scp -r  socialsecurity.git tim@reposerver:/home/tim/
tim@reposerver's password: 
scp: realpath /home/tim/socialsecurity.git: No such file
scp: upload "/home/tim/socialsecurity.git": path canonicalization failed
scp: failed to upload directory socialsecurity.git to /home/tim/

Fix: add a “-O” argument to your “scp” command line:

The -O tells scp to use the Original algorithm.

tim@timub2404:~/eclipse-work$ scp -r -O socialsecurity.git tim@reposerver:/home/tim/
tim@reposerver's password: 
HEAD                                          100%   21    11.5KB/s   00:00    
0b582a00b112f748f21bb23ffa752914571733        100%  824   319.8KB/s   00:00    
fc089c9ada51bd2c992218e71b647639bd1120        100%  845   612.1KB/s   00:00    
  <<snip unhelpful>> 
config                                        100%  136    98.3KB/s   00:00    

Do you care why you need “-O”? I know I don’t. Ubuntu 24.04 upgraded its version of scp, so now you need it. The current man page says “The legacy SCP protocol (selected by the -O flag) requires execution of the remote user’s shell to perform glob(3) pattern matching. This requires careful quoting of any characters that have special meaning to the remote shell, such as quote characters.”

Which, for the above example, is just incorrect. The command line has no “~” or anything else that requires glob() pattern matching.

BTW – another valid work-around is to put your directory into a single .tgz file, and scp that without the -r command line argument.

Posted in Software Project, Ubuntu | Comments Off on SSH path canonicalization failed

Fix for white label error in tutorial

For those of you who are following https://learnk8s.io/spring-boot-kubernetes-guide (from 2022)

When you get the dreaded “Whitelabel Error Page” and you are pulling out your hair, here is your solution:

The stack overflow: https://stackoverflow.com/questions/42330870/spring-boot-unable-to-resolve-freemarker-view (from 2020)

Which points you to https://github.com/spring-projects/spring-boot/issues/15131 (from 2018)

And the answer is:

Rename src/main/resources/templates/index.ftl to src/main/resources/templates/index.ftlh

Keywords: Spring Boot SpringBoot Freemarker Kubernetes spring-boot-starter-freemarker

Posted in Software Project | Comments Off on Fix for white label error in tutorial

Most Obvious AI Quote

… that argues reasoning models were most likely mimicking the data they saw in training rather than actually solving new problems.

2024-12-21, The Wall Street Journal, “The Next Great Leap in AI is behind schedule and crazy expensive”

At least the subheading got it right (“behind schedule and crazy expensive”). But really, does anybody believe that AI is actually solving new problems? That it is doing something more than just mimicking its inputs? AI researchers are just building ELIZA with more hidden layers, so nobody can tell exactly what is going to happen.

But, that was in the 1960s, so I’m guessing that didn’t happen for them, and in any case just doesn’t matter for their “high-complexity” models.

Posted in Machine Learning | Comments Off on Most Obvious AI Quote

Intel 14700K

This is a new computer build.

Some facts on the CPU: it is currently #89 on PassMark [53,737] cpubenchmark.net, with a turbo speed of 5.6 GHz and TDP of 125W to 253W. It is 20 cores/28 threads, first seen Q4 2023. 8 of those are “Performance Cores”, 12 are “Efficient Cores”. It is #2 on single-thread rating [4,484] behind the i9-13900K at [4,641].

ItemProductCost
CPUIntel Core i7 14700K 3.4GHz LGA 1700 125W$399
CoolerCorsair iCUE H150i Elite LCD XT Liquid, AF120 RGB Fans, 360mm Radiator$249
RAMG.SKILL Trident Z5 RGB 64GB (2x32GB) DDR5 5600$204
MotherboardASUS ROG STRIX Z790-A Gaming Wifi LGA 1700, 2.5Gb LAN, Wi-Fi 7 MLO, PCIe 5.0 16x slot, 5x M.2 PCIe 4.0 mode$359
Power SupplySeasonic FOCUS GX-850 80+ Gold, fully modular, fan control, ATX12V, EPS12V$134
VideoGIGABYTE GeForce RTX 4070 Ti Super AERO OC 16GB GDDR6X, PCIe x4.0, DLSS 3, 8,448 CUDA cores$880
CaseCorsair iCUE 5000D RGB Mid-Tower, True White 3x AF120 Fans$219
SS DriveSamsung 990 Pro 4TB PCIe Gen 4×4 NVMe, 7450MB/sec, M2.2280 ($339 on 2024/mar)$249
HD Drivenone
DVD/CDnone
OSWindows 11 Professional 64 bit OEM no media$169
Keyboard
Mouse
Total$2,862
Posted in Computer Builds | Comments Off on Intel 14700K

Better pattern for @Autowired

Just ran across this in a video about Spring Boot by Frank Moley and I thought it deserved a callout.

When creating your Beans (e.g. Controller), you can either:

@RestController() public class ItemController { @Autowired; private ItemService itemService;

Or you can do this:

@RestController() public class ItemController { private final ItemService itemService; @Autowired; public ItemController(ItemService itemService) { this.itemService = itemService }

Both work, but the second one makes JUnit testing way easier. The first one @Autowires the actual field, and is really hard to isolate for test. The second one lets the field stay POJO, and @Autowires the constructor. Both are Spring friendly, but only the second one is JUnit friendly.

Posted in Software Project, Uncategorized | Comments Off on Better pattern for @Autowired

Argo CD notes

Argo CD (github source) is a Continuous Delivery tool done “GitOps” style. Here, that means keeping all of your (Kubernetes) application definitions and configurations under source code control (git).

Assuming you follow GitOps CI/CD best practices of separate repositories (one for application code and a second for your YAML configurations), your development process is likewise split into two discrete problems:

  1. Application Code – build the container images (CI)
  2. Configuration Code – keep your (Kubernetes) configuration files synchronized with the repository (CD)

Argo CD solves the second bullet. And does it pretty well. It supports writing your configuration code in any of: plain YAML, Helm charts. Kustomize, and Jsonnet.

It also supports enforcing that all Kubernetes changes “from the command line” are at least detected (and if desired, automatically rolled back). E.g. if some rogue administrator issues a “kubectl scale –replicas=3 …” command, Argo CD will at least change the status to “Out of Sync”, and is capable of automatically re-synchronizing, and therefore un-doing the change.

Reference: compare Argo CD versus Tekton versus Jenkins X

Question: do you need separate repositories or just separate directories? i.e. Argo CD takes a repository and a path to configure an “application” – does it ignore commits to files outside of the directory? Answer: yes, it ignores them, so no, you don’t technically need separate repositories. Which means all of the Monorepo fans out there are in luck.

Posted in Software Engineering | Comments Off on Argo CD notes