Adventures in Flooring

I first ventured into the realm of home improvement by mistake. I was cleaning my new fangled condo with 409 and Baking Soda, the enemy of dirty grout lines. As I scrubbed, I destroyed the sealant between the shower and the floor, revealing quite a few interesting things. First, the floor started to curl up, which made me confused. Wasn't floor supposed to be wood or tile,  how could it possibly curl up? I pulled back the floor some more and saw that the floor was actually like wall paper! Next, when I stepped onto the area where the floor was curling, I would sink into the floor, much like stepping into a bed. What was going on! Into the rabbit hole we go.

ikeaConfused.jpg

Apparently floors have three layers The top layer, what you see, can be made of multiple different materials. The first layer is what you step on when you go take a shower. It can be either Vinyl, Hardwood, or Tile. One layer underneath that, the middle layer, is what your top layer rests on. Usually this is plywood or backer board. Lastly, underneath all that is the subfloor, which hooks into joists, which actually hold up the building. What actually happened to my floor was that my 409 / Baking Soda simply unveiled all the damage underneath and into the middle layer. Here's a diagram of a floor:

flooring.png

The middle layer, made of plywood, which normally held up my floor was corroded by mold and water. My simple $50 fix became a $1000 fix, oh yay for home ownership! Here's a picture of the damaged floor:

plywood.png

There are a few fixes depending on how much money you want to spend, and like the n00b I was, I spent a lot of time in Home Depot dazed and confused. BTW, Home Depot's customer service is amazing, they are really patient and used to stupid people looking at a piece of equipment with a lost gaze. This is what normal people feel like when they walk into Frys!

The fixes are two fold, assuming the sub floor (the bottom layer), isn't damaged. Because bathrooms have lots of water, installing plywood is the "cheap" choice, but the plywood will eventually become water soaked and damaged over time. Have you ever hiked in the mountains and seen water soaked branches that are fragile and destroyed? That happens to your floor! You can buy redguard, which coats the plywood with a rubbery overcoat that resists the water, leaving a pool on top of the plywood preventing damage from the water. However, the water has to go somewhere and it just soaks into your top layer. If your top layer is water proof, this might not be a problem, but then you need tile (which we'll get into).

Another option, is laying down cement, which servers the same function as plywood but with cement instead. This protects your subfloor but you still have the water on the top layer problem. What Home Depot recommended was Backer Board, which absorbs water preventing it from falling down. Now you can buy these pieces of backer board and put water resistant tape between the seams of the backer board, creating a water absorbing middle layer. Now we have a middle layer, what to put on top?

cementBoard.jpg

There seems to be four options depending on how much money you want to spend. First, you can install natural hardwood, but that absorbs water and is a bad choice for kitchens and bathrooms. Second, you can install vinyl or laminoid, which is essentially wallpaper for your middle layer. The materials can be water resistant, preventing water from soaking into the middle layer at the high end of vinyl. The third option is tile, which is water proof, but is more expensive to install. Each tile piece has to be cut, laid down, and filled with grout (which is a pain to clean!) between each piece. And finally, you can install natural stone if you just IPOed your company.

I went with option two, installing vinyl on the top layer. What we had and originally caused the problem was a layer of vinyl on top of the middle layer of plywood, followed by the subfloor. The plywood was damaged by moisture, which happens in every bathroom. We didn't touch the subfloor. We replaced the middle layer with water absorbing backer board and put a layer of vinyl on top. Here's the subfloor after throwing the other layers away:

subFloor.jpg

Step one, how do we install the backer board onto the sub floor? First, you have to saw the backer board into the correct measurements for your bathroom. My bathroom involved two pieces of backer board, sealed between them with some water sealant. Each backer board is screwed into the subfloor with some nails. Next, you use the sealant along all the edges of the bathroom. The sealant prevents water from going into the subfloor along the small seams at the edge between the wall and the backer board.

subfloorInstalled.jpg

Step two, how do we install the vinyl on top of the backer board? I bought this floor that claims to be the easiest floor ever to install. Each vinyl piece is like a puzzle that snaps together to form a whole floor. Each piece is cut using a utility knife. What really happens is that each vinyl piece is water resistant and made of very strong material, so strong that it had to be cut with a saw. The actual layering of each piece to snap together worked fairly well, but the whole floor is "floating". There is adhesive behind each vinyl piece that sticks the vinyl to the backer board, but there is nothing solid holding the floor together like there is with tile. I've read reports that after a while, the floor's joints become weak and there are open seams between the vinyl pieces. We'll see what happens soon.

Step three, can I finally shower? Once the vinyl is put on top of the backer board, you seal up the edges of the bathroom to prevent water from dripping below the vinyl. Thus, each layer of the floor except the subfloor are individually sealed along the edges for water protection. The sealant takes a while to dry, so I was told to not use the bathroom for a few days. I waited 3 days and now I can finally shower!

El Fin

Becoming a Technical Leader

I forget where I stumbled upon this book, but I'm happy I did. Most leadership books have a tone of superiority, almost with a sense of being talked down to. What I liked most about this book was that it was easy to read and understand, while at the same time imparting valuable lessons and bringing up interesting questions.

Instead of the usual leadership book theme of lead the charge, understanding organizational issues and how to hire people, Becoming a Technical Leader actually starts with the reader. It asks the question, why do you, as a person, want to become a leader? What makes you actually valuable to an organization and why would anyone in their right mind actually follow or listen to anything you say? Every chapter has some central tenet that actually forces the reader to become introspective, with a list of questions at the end of the chapter to reinforce and clear out ideas. It's like Math homework without a correct answer in the back to the chapter.

For example, one chapter examines how people grow in skill and as a person. We all like to imagine that growth if linear, like y = mx + b, or if for some period of time, exponential growth. I felt the exponential growth curve when I first entered graduate school, feeling like a complete idiot  but at the same time learning so much everyday that I was on overdrive. The first time I was ever able to build a JIT compiler, assembling the add eax, ebx instruction was such a miracle. It felt exciting to learn so much and accomplish so much. However, by the time I wrote my fourth JIT compiler, I felt a plateau, and needed some stimulus to find new challenges. This book details the growth trajectory so I learned what signals to look out for at each step of the growth stage. 

However, the best part of the book is simply how easy it is to learn something about myself. When some discussions come up, I become defensive and have to become aware of my defense and shut it down. Others, like a good friend, disagreements come easy and no hard feelings erupt. It's simply a trained muscle of socially grace. That's what's so amazing about this book. I respect the author as a leader through his writing, as he tries to give me constructive feedback on how to improve myself. 

That's really the crux of the book. Instead of being a another outward looking leadership book, this book forces introspection on why and what to improve in yourself to become a leader. Not only did I learn about leadership skills, I learned a lot about myself. Highly recommended.

Solaris 11 Colored Terminal

I'm starting to do some development with Solaris, and while most of it is linux like, some things are still a little different. I spent a good 20 minutes trying to figure out how to add colors to the terminal output. No colors = No fun! Most of the googling showed a large array of export LS_COLORS="color combination", but that didn't work. All my previous .bashrc files that worked on Linux didn't work on Solaris. In addition, ls --color=auto didn't even show any colors. I was confused!

Apparently all I needed to add was:

export TERM=xterm-color

Now I get all my colorful glory in Solaris!

Thread Local Storage in Object Files

I was rummaging through some generated x86-64 assembly compiled by Clang and Intel's C compiler (ICC), and I noticed something strange. In Clang generated code, there are a large number of loads from a _DYNAMIC section.

1
6a1:	lea 0x2002d7(%rip),%rdi        # 200980 <_DYNAMIC+0x1a8>

The assembly didn't bother me, so much as the comment. What is the _DYNAMIC section doing? I've never seen anything like it before and it made me wonder. This was especially confusing because these instructions didn't seem to occur with the code compiled by ICC. What is that Dynamic section?

At first, I thought it was simply a case of position independent code (PIC) because the load was offset from the instruction pointer (%rip). I don't remember explicitly enabling position independent code, but I thought maybe Clang compiles PIC code by default. But something else struck me as odd. The ICC generated code had loads relative from %rip as well, but no comment. Maybe I just forgot to build the binary with debug information and that would add the _DYNAMIC comment?

I recompiled the binary with debug information with ICC but that didn't do it either. I knew that there was probably some issue with Thread Local Storage because I've experienced it before. I set out to discover what it was. Let's have some simple C code with a thread local variable and see what happens.

1
2
3
4
5
6
__thread int _threadInt; 

int foo() {
    printf("Thread int is: %d\n", _threadInt);
    return 0;
}

Clang generates the following assembly code:: 

1
2
3
4
5
6
7
8
 push   %rax
 mov    %fs:0xfffffffffffffffc,%esi
 mov    $0x400658,%edi
 xor    %al,%al
 callq  4003f0 <printf@plt>
 xor    %eax,%eax
 pop    %rdx
 retq   

Here, instruction #2 is the most important. This is a thread local variable access. The generated assembly uses the initial-exec storage model, which is another blog post. Instruction #5 is the call to printf. Here the assembly makes sense. We just load the thread local variable and pass it as a parameter into printf. (X86-64 passes parameters in registers). But there's still no load from _DYNAMIC! What's going on? I read more about the DYNAMIC section in ELF files [1], trying to figure out where the thread local variable is allocated. Let's take a look at the DYNAMIC section in the object file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
objdump -x main.o 

Dynamic Section:
  NEEDED               libc.so.6
  NEEDED               ld-linux-x86-64.so.2
  INIT                 0x0000000000000568
  FINI                 0x0000000000000718
  GNU_HASH             0x00000000000001b8
  STRTAB               0x0000000000000360
  SYMTAB               0x00000000000001f8
  STRSZ                0x00000000000000b6
  SYMENT               0x0000000000000018
  PLTGOT               0x0000000000200990
  PLTRELSZ             0x0000000000000060
  PLTREL               0x0000000000000007
  JMPREL               0x0000000000000508
  RELA                 0x0000000000000478
  RELASZ               0x0000000000000090
  RELAENT              0x0000000000000018
  VERNEED              0x0000000000000438
  VERNEEDNUM           0x0000000000000002
  VERSYM               0x0000000000000416
  RELACOUNT            0x0000000000000001

Here the dynamic section refers to elements that are required during dynamic linking. However, looking at this, we see some sections in ELF describing what's needed at runtime. We need libc for printf. We also see a symbol table (Item 10), but still nothing here that tells us how the thread local variable is accessed. In addition, the generated code doesn't have a comment indicating a load from a DYNAMIC section. But this gave me an idea, what if we made the thread local variable extern so the linker would have to resolve the address later? And since it should be in the dynamic section, let's make the code a shared library. Let's write some code:

1
2
3
4
5
6
7
8
clang -fpic main.c -O -shared -o main.o

extern __thread int _threadInt;

int foo() {
    printf("Thread int is: %d\n", _threadInt);
    return 0;
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
push   %rax
lea 0x20031f(%rip),%rdi        # 2009a8 <_DYNAMIC+0x1a8>
callq 598 <__tls_get_addr@plt>
mov    (%rax),%esi
lea    0xac(%rip),%rdi        # 746 <_fini+0xe>
xor    %al,%al
callq  578 <printf@plt>
xor    %eax,%eax
pop    %rdx
retq   
nopw %cs:0x0(%rax,%rax,1)

Alright, finally! We have a reference to the DYNAMIC section again! Why did this happen? Compilers can optimize Thread Local Storage using different thread local storage models. One optimization is if the compiler can determine that the thread local variable is only referenced within the executable, a level of indirection can be optimized away, hence the lack of a _DYNAMIC access. By making the binary a shared library, we prevented the compiler from performing this optimization. So now that we finally have a reference to _DYNAMIC, what is it actually referencing? We see something interesting about instruction #3, a call to tls_get_addr@plt. 

This is one method of accessing thread local storage [2]. The code calls a function that retrieves the address of the local variable through the Procedure Linking Table (PLT). How the PLT works is another long blog post. Essentially, thread local access uses a table to patch up relocation points to access variables. Code accesses the variable through this table. Then I thought, oh let's take a look at the dynamic relocation entries in the object file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
readelf -r main.o

Relocation section '.rela.dyn' at offset 0x478 contains 6 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000002007f8  000000000008 R_X86_64_RELATIVE                    00000000002007f8
000000200990  000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
000000200998  000400000006 R_X86_64_GLOB_DAT 0000000000000000 _Jv_RegisterClasses + 0
0000002009a0  000500000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize + 0
0000002009a8  000700000010 R_X86_64_DTPMOD64 0000000000000000 _threadInt + 0
0000002009b0  000700000011 R_X86_64_DTPOFF64 0000000000000000 _threadInt + 0
Aha! There it is, Line #9. If we look at the comment in the disassembled code, we see that it states an entry in the _DYNAMIC section pointing to address 0x2009a8. When we look at the dynamic relocation section, we see our thread local variable at the exact offset. 

Whew, at least now I know what the _DYNAMIC section is for. If refers to the dynamic relocation section in an ELF file, not the dynamic shared library section. Another blog post will have to be written to describe how relocation works and each thread local variable access optimization strategies.

References:
[1] Executable and Linkable Format
[2] ELF Handling for Thread Local Storage
[3] Eli Bendersky's Position Independent Code

Research and Engineering

I'm running on the current theory that the major the major distinction between Research and Engineering is determining precise enough. For most of the engineering work I've done, we never really needed answers with really high precision. An 80% confidence interval with a 80 - 90% approximate answer was good enough. We could act on it, build in some tolerance, and be on our merry way. It allowed us to move fast and generally not get stuck in holes deep diving to find out exactly why something was happening. We had fairly high confidence some effect was happening and how to solve it. For example, if the system profiler states there is a large amount of CPU time spent in some function, we optimize it, measure it, and call it a day.

Science and Research on the other hand, isn't satisfied with 80% approximations. Research demands high precision, with 99% confidence intervals. It's almost impossible to state anything as fact with 100% confidence, but Research and Science inches closer and closer to that precision. Science exists to determine facts, characterize facts, and understand what's happening in a system. If the system profiler states a large amount of CPU time is spent in some function, Science wants to understand why it's spending a lot of time. What's happening to the CPU, to the cache lines, is it an expensive CPU instruction, then optimize it and characterize it to a much further detail. What engineers call holes, Researchers call characterization.

However, this divide doesn't have to exist. The best engineering and the best research become unified. Apple engineers their iPhones to intense micrometer precision. Researchers can detail and explain every single small aspect about their design and why design decisions were made. Facebook engineers their web services breaking the boundaries about what was normal scale, creating new technologies to help them do that. The browser wars used Self VM research to optimize their Javascript engines. I think the two fields climb towards the same goals but from different angles. Engineering builds an artifact as quickly and reliably as possible. Research tries to understand every step of the way towards a new discovery. The best projects do both.