Check server response of
Server response
NS records
Whois domain
301 Moved Permanently - devth.com
HTTP Status: 301
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; [email protected])
Date: Wed, 30 Apr 2025 08:19:40 GMT
Content-Type: text/html
Content-Length: 167
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Wed, 30 Apr 2025 09:19:40 GMT
Location: https://devth.com/
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=EebFRtbtwcdgHJI9q57N7CVNiDRARou%2BoHZQt4%2FjrpI0AGFls6DzfoeweNlxkkfWL3Vy%2BPN5YFJl8FSOswYGs%2FAs7rIRittgcJPnNJiQLgy%2BQeCqqxt4KIA1ZcbuPKkAap%2BlCZWs6x8%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 93859d94bdb76ded-CPH
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=16045&min_rtt=16045&rtt_var=8022&sent=1&recv=3&lost=0&retrans=0&sent_bytes=0&recv_bytes=165&delivery_rate=0&cwnd=249&unsent_bytes=0&cid=0000000000000000&ts=0&x=0"
HTTP Code 301 Moved Permanently
301 status code means that the requested resource has been permanently moved to a new URL. All future requests should use the new address.
When is Code 301 used?
- When changing a website domain
- When modifying URL structures
- When setting up redirects for SEO
What does Code 301 mean for the user?
The browser will automatically redirect the user to the new address, and search engines will update their indexes.
200 OK - https://devth.com/
HTTP Status: 200
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; [email protected])
Date: Wed, 30 Apr 2025 08:19:41 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
access-control-allow-origin: *
Age: 381245
Cache-Control: public, max-age=0, must-revalidate
content-disposition: inline
last-modified: Fri, 25 Apr 2025 22:25:35 GMT
strict-transport-security: max-age=63072000
x-matched-path: /
x-vercel-cache: HIT
x-vercel-id: arn1::422h2-1746001181053-ac3f113cb30a
cf-cache-status: DYNAMIC
vary: accept-encoding
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=QfJAtxhcnMQxCcAnuV13WE%2FceSXVJql0muhtgehzfNjfGe1FY18MN35cRUO951gn3MP%2By%2B5ueAzYVwr6WtMRaFi41IsvnY8k7Bz23%2Fj6abGU7aC0cIH5cubL6iTttcayN1fX5263z3g%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 93859d954ea8d090-CPH
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=16052&min_rtt=16000&rtt_var=4537&sent=5&recv=7&lost=0&retrans=0&sent_bytes=2832&recv_bytes=784&delivery_rate=252324&cwnd=252&unsent_bytes=0&cid=f28f44ff643599a1&ts=93&x=0"
HTTP Code 200 OK
200 status code is a standard successful HTTP server response. It means that the client’s request (e.g., from a browser) was successfully processed, and the server is delivering the requested data.
When is Code 200 used?
- When loading a web page
- When successfully receiving an API response
- When processing a form or another HTTP request
What does Code 200 mean for the user?
The user receives content without errors, and the page or application functions properly. If Code 200 is accompanied by data, the browser or program processes and displays it to the user.
GET / HTTP/1.1 Host: devth.com Accept: */* User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; [email protected])
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8" data-next-head=""/><meta name="viewport" content="initial-scale=1, width=device-width" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="57x57" href="/images/favicon/apple-touch-icon-57x57.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="114x114" href="/images/favicon/apple-touch-icon-114x114.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="72x72" href="/images/favicon/apple-touch-icon-72x72.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="144x144" href="/images/favicon/apple-touch-icon-144x144.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="60x60" href="/images/favicon/apple-touch-icon-60x60.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="120x120" href="/images/favicon/apple-touch-icon-120x120.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="76x76" href="/images/favicon/apple-touch-icon-76x76.png" data-next-head=""/><link rel="apple-touch-icon-precomposed" sizes="152x152" href="/images/favicon/apple-touch-icon-152x152.png" data-next-head=""/><link rel="icon" type="image/png" href="/images/favicon/favicon-196x196.png" sizes="196x196" data-next-head=""/><link rel="icon" type="image/png" href="/images/favicon/favicon-96x96.png" sizes="96x96" data-next-head=""/><link rel="icon" type="image/png" href="/images/favicon/favicon-32x32.png" sizes="32x32" data-next-head=""/><link rel="icon" type="image/png" href="/images/favicon/favicon-16x16.png" sizes="16x16" data-next-head=""/><link rel="icon" type="image/png" href="/images/favicon/favicon-128.png" sizes="128x128" data-next-head=""/><meta name="theme-color" content="#333"/><link rel="shortcut icon" href="/favicon.ico"/><meta property="og:title" content="devth - Trevor Hartman"/><meta property="og:description" content="The blog of Trevor Hartman"/><meta property="og:type" content="website"/><meta property="article:author" content="Trevor Hartman"/><meta name="emotion-insertion-point" content=""/><style data-emotion="mui-style-global 0"></style><style data-emotion="mui-style-global w82362">::selection{background:rgba(255, 255, 102, 0.4);}:root{--ch-0:light;--ch-1:#6e7781;--ch-2:#0550ae;--ch-3:#953800;--ch-4:#24292f;--ch-5:#8250df;--ch-6:#116329;--ch-7:#cf222e;--ch-8:#0a3069;--ch-9:#82071e;--ch-10:#f6f8fa;--ch-11:#ffebe9;--ch-12:#dafbe1;--ch-13:#ffd8b5;--ch-14:#eaeef2;--ch-15:#57606a;--ch-16:#ffffff;--ch-17:#eaeef280;--ch-18:#fdff0033;--ch-19:#1a85ff;--ch-20:#add6ff;--ch-21:#0969da;--ch-22:#f6f8fa;--ch-23:#d0d7de;--ch-24:#8c959f;--ch-25:#afb8c133;--ch-26:#ffffffe6;}</style><style data-emotion="mui-style-global 1mwnf52">html{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;box-sizing:border-box;-webkit-text-size-adjust:100%;}*,*::before,*::after{box-sizing:inherit;}strong,b{font-weight:700;}body{margin:0;color:rgba(0, 0, 0, 0.87);font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.1428571428571428rem;line-height:1.5;background-color:#fff;}@media (min-width:600px){body{font-size:1.1667rem;}}@media (min-width:900px){body{font-size:1.3333rem;}}@media (min-width:1200px){body{font-size:1.3333rem;}}@media print{body{background-color:#fff;}}body::backdrop{background-color:#fff;}blockquote{border-left:4px solid #ddd;margin-left:0;padding-left:1rem;}</style><style data-emotion="mui-style-global 19i3dl6">p{font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.1428571428571428rem;line-height:1.5;}@media (min-width:600px){p{font-size:1.1667rem;}}@media (min-width:900px){p{font-size:1.3333rem;}}@media (min-width:1200px){p{font-size:1.3333rem;}}code{font-family:monospace;background-color:#f4f4f4;border-radius:8px;padding:2px 8px;}</style><style data-emotion="mui-style 9wvnva 1xkh7h6 m89tgr 1x4jos1 v8mqvs 1hk2sn0 1d3bbye z689d8 2mwbgk 13a8w4s 628fjv d04u73 1txkj0z idv8vo 17gjue1 1s6qh0k yircub 1o57n5v 198olr7 1v16n3n 1kjbk98 mpnz38 cq9yas 1x1zu4f">.mui-style-9wvnva{width:100%;margin-left:auto;box-sizing:border-box;margin-right:auto;padding-left:16px;padding-right:16px;}@media (min-width:600px){.mui-style-9wvnva{padding-left:24px;padding-right:24px;}}@media (min-width:1200px){.mui-style-9wvnva{max-width:1200px;}}.mui-style-1xkh7h6{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;-webkit-align-items:baseline;-webkit-box-align:baseline;-ms-flex-align:baseline;align-items:baseline;}.mui-style-1xkh7h6>:not(style):not(style){margin:0;}.mui-style-1xkh7h6>:not(style)~:not(style){margin-left:16px;}.mui-style-m89tgr{margin:0;font:inherit;line-height:inherit;letter-spacing:inherit;color:#333;-webkit-text-decoration:underline;text-decoration:underline;text-decoration-color:var(--Link-underlineColor);--Link-underlineColor:rgba(51, 51, 51, 0.4);font-size:70px;font-weight:700;-webkit-transition:all .3s ease-out 0s,font-size 0.5s ease-out 0.5s;transition:all .3s ease-out 0s,font-size 0.5s ease-out 0.5s;color:#ccc;-webkit-text-decoration:none;text-decoration:none;cursor:pointer;}.mui-style-m89tgr:hover{text-decoration-color:inherit;}.mui-style-m89tgr:hover{color:black;}.mui-style-1x4jos1{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.mui-style-1x4jos1>:not(style):not(style){margin:0;}.mui-style-1x4jos1>:not(style)~:not(style){margin-left:16px;}.mui-style-v8mqvs{margin:0;font:inherit;line-height:inherit;letter-spacing:inherit;color:#333;-webkit-text-decoration:underline;text-decoration:underline;text-decoration-color:var(--Link-underlineColor);--Link-underlineColor:rgba(51, 51, 51, 0.4);cursor:pointer;-webkit-transition:color .3s ease-in-out,box-shadow 2s ease-out;transition:color .3s ease-in-out,box-shadow 2s ease-out;-webkit-text-decoration:none;text-decoration:none;font-family:monospace;color:red;}.mui-style-v8mqvs:hover{text-decoration-color:inherit;}.mui-style-v8mqvs:hover{color:black;}.mui-style-1hk2sn0{width:100%;margin-left:auto;box-sizing:border-box;margin-right:auto;padding-left:16px;padding-right:16px;margin-top:32px;}@media (min-width:600px){.mui-style-1hk2sn0{padding-left:24px;padding-right:24px;}}@media (min-width:1200px){.mui-style-1hk2sn0{max-width:1200px;}}.mui-style-1d3bbye{box-sizing:border-box;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;width:100%;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;}.mui-style-z689d8{box-sizing:border-box;margin:0;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;}@media (min-width:600px){.mui-style-z689d8{-webkit-flex-basis:8.333333%;-ms-flex-preferred-size:8.333333%;flex-basis:8.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:8.333333%;}}@media (min-width:900px){.mui-style-z689d8{-webkit-flex-basis:8.333333%;-ms-flex-preferred-size:8.333333%;flex-basis:8.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:8.333333%;}}@media (min-width:1200px){.mui-style-z689d8{-webkit-flex-basis:8.333333%;-ms-flex-preferred-size:8.333333%;flex-basis:8.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:8.333333%;}}@media (min-width:1536px){.mui-style-z689d8{-webkit-flex-basis:8.333333%;-ms-flex-preferred-size:8.333333%;flex-basis:8.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:8.333333%;}}.mui-style-2mwbgk{box-sizing:border-box;margin:0;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-flex-basis:100%;-ms-flex-preferred-size:100%;flex-basis:100%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:100%;}@media (min-width:600px){.mui-style-2mwbgk{-webkit-flex-basis:83.333333%;-ms-flex-preferred-size:83.333333%;flex-basis:83.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:83.333333%;}}@media (min-width:900px){.mui-style-2mwbgk{-webkit-flex-basis:83.333333%;-ms-flex-preferred-size:83.333333%;flex-basis:83.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:83.333333%;}}@media (min-width:1200px){.mui-style-2mwbgk{-webkit-flex-basis:83.333333%;-ms-flex-preferred-size:83.333333%;flex-basis:83.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:83.333333%;}}@media (min-width:1536px){.mui-style-2mwbgk{-webkit-flex-basis:83.333333%;-ms-flex-preferred-size:83.333333%;flex-basis:83.333333%;-webkit-box-flex:0;-webkit-flex-grow:0;-ms-flex-positive:0;flex-grow:0;max-width:83.333333%;}}.mui-style-13a8w4s{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:column;-ms-flex-direction:column;flex-direction:column;padding:6px 16px;-webkit-box-flex:1;-webkit-flex-grow:1;-ms-flex-positive:1;flex-grow:1;padding:8px;}.mui-style-13a8w4s .MuiTimelineItem-root:before{-webkit-flex:0;-ms-flex:0;flex:0;padding:0px;}.mui-style-628fjv{list-style:none;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;position:relative;min-height:70px;}.mui-style-628fjv::before{content:"";-webkit-flex:1;-ms-flex:1;flex:1;padding:6px 16px;}.mui-style-d04u73{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:column;-ms-flex-direction:column;flex-direction:column;-webkit-flex:0;-ms-flex:0;flex:0;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.mui-style-1txkj0z{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-self:baseline;-ms-flex-item-align:baseline;align-self:baseline;border-style:solid;border-width:2px;padding:4px;border-radius:50%;box-shadow:none;margin:11.5px 0;background-color:transparent;border-color:#333;}.mui-style-idv8vo{width:2px;background-color:#bdbdbd;-webkit-box-flex:1;-webkit-flex-grow:1;-ms-flex-positive:1;flex-grow:1;}.mui-style-17gjue1{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.1428571428571428rem;line-height:1.5;-webkit-flex:1;-ms-flex:1;flex:1;padding:6px 16px;text-align:left;margin-top:-8px;}@media (min-width:600px){.mui-style-17gjue1{font-size:1.1667rem;}}@media (min-width:900px){.mui-style-17gjue1{font-size:1.3333rem;}}@media (min-width:1200px){.mui-style-17gjue1{font-size:1.3333rem;}}.mui-style-1s6qh0k{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:500;font-size:1.3035714285714286rem;line-height:1.6;color:red;font-weight:700;font-family:monospace;}@media (min-width:600px){.mui-style-1s6qh0k{font-size:1.4063rem;}}@media (min-width:900px){.mui-style-1s6qh0k{font-size:1.5625rem;}}@media (min-width:1200px){.mui-style-1s6qh0k{font-size:1.5625rem;}}.mui-style-yircub{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:bold;font-size:2rem;line-height:1.2;text-wrap:balance;display:inline;}@media (min-width:600px){.mui-style-yircub{font-size:2.5rem;}}@media (min-width:900px){.mui-style-yircub{font-size:2.7083rem;}}@media (min-width:1200px){.mui-style-yircub{font-size:2.9167rem;}}.mui-style-1o57n5v{margin:0;font:inherit;line-height:inherit;letter-spacing:inherit;color:#333;-webkit-text-decoration:underline;text-decoration:underline;text-decoration-color:var(--Link-underlineColor);--Link-underlineColor:rgba(51, 51, 51, 0.4);cursor:pointer;-webkit-transition:all .8s ease-out,padding-bottom .2s ease-out;transition:all .8s ease-out,padding-bottom .2s ease-out;-webkit-text-decoration:none;text-decoration:none;display:inline-block;text-wrap:balance;position:relative;padding-bottom:0px;}.mui-style-1o57n5v:hover{text-decoration-color:inherit;}.mui-style-1o57n5v::after{content:'';position:absolute;width:100%;-webkit-transform:scaleX(0);-moz-transform:scaleX(0);-ms-transform:scaleX(0);transform:scaleX(0);border-radius:0px;height:8px;bottom:0;left:0;background:#FFCC0033;transform-origin:bottom right;-webkit-transition:-webkit-transform 0.45s ease-out;transition:transform 0.45s ease-out;}.mui-style-1o57n5v:hover::after{-webkit-transform:scaleX(1);-moz-transform:scaleX(1);-ms-transform:scaleX(1);transform:scaleX(1);transform-origin:bottom left;}.mui-style-1o57n5v:hover{padding-bottom:0px;}.mui-style-198olr7{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.1428571428571428rem;line-height:1.75;color:#666;font-size:1rem;}@media (min-width:600px){.mui-style-198olr7{font-size:1.2857rem;}}@media (min-width:900px){.mui-style-198olr7{font-size:1.2857rem;}}@media (min-width:1200px){.mui-style-198olr7{font-size:1.2857rem;}}.mui-style-1v16n3n{width:100%;margin-left:auto;box-sizing:border-box;margin-right:auto;padding-left:16px;padding-right:16px;padding:80px;margin-top:32px;}@media (min-width:600px){.mui-style-1v16n3n{padding-left:24px;padding-right:24px;}}@media (min-width:1200px){.mui-style-1v16n3n{max-width:1200px;}}.mui-style-1kjbk98{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:column;-ms-flex-direction:column;flex-direction:column;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.mui-style-1kjbk98>:not(style):not(style){margin:0;}.mui-style-1kjbk98>:not(style)~:not(style){margin-top:9.6px;}.mui-style-mpnz38{margin:0;font:inherit;line-height:inherit;letter-spacing:inherit;color:#333;-webkit-text-decoration:underline;text-decoration:underline;text-decoration-color:var(--Link-underlineColor);--Link-underlineColor:rgba(51, 51, 51, 0.4);cursor:pointer;-webkit-transition:all .5s ease-in-out;transition:all .5s ease-in-out;color:red;font-family:monospace;-webkit-text-decoration:none;text-decoration:none;text-transform:lowercase;}.mui-style-mpnz38:hover{text-decoration-color:inherit;}.mui-style-mpnz38:hover{color:black;}.mui-style-cq9yas{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.0625rem;line-height:1.43;margin-left:auto;}@media (min-width:600px){.mui-style-cq9yas{font-size:1.049rem;}}@media (min-width:900px){.mui-style-cq9yas{font-size:1.049rem;}}@media (min-width:1200px){.mui-style-cq9yas{font-size:1.049rem;}}.mui-style-1x1zu4f{margin:0;font-family:"Helvetica Neue",-apple-system;font-weight:400;font-size:1.0625rem;line-height:1.43;color:grey;}@media (min-width:600px){.mui-style-1x1zu4f{font-size:1.049rem;}}@media (min-width:900px){.mui-style-1x1zu4f{font-size:1.049rem;}}@media (min-width:1200px){.mui-style-1x1zu4f{font-size:1.049rem;}}</style><link rel="preload" href="/_next/static/css/e2c125283e972dd6.css" as="style"/><link rel="stylesheet" href="/_next/static/css/e2c125283e972dd6.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-42372ed130431b0a.js"></script><script src="/_next/static/chunks/webpack-9c8665dc40a3cb46.js" defer=""></script><script src="/_next/static/chunks/framework-b96261d959dd50e7.js" defer=""></script><script src="/_next/static/chunks/main-e5cda668ab60045f.js" defer=""></script><script src="/_next/static/chunks/pages/_app-f489dbe814a55aa4.js" defer=""></script><script src="/_next/static/chunks/58-7616dc1447366fe5.js" defer=""></script><script src="/_next/static/chunks/365-4522171229723362.js" defer=""></script><script src="/_next/static/chunks/pages/index-db1bbab022f902a1.js" defer=""></script><script src="/_next/static/txPjj0UlODJm11FYMEzVo/_buildManifest.js" defer=""></script><script src="/_next/static/txPjj0UlODJm11FYMEzVo/_ssgManifest.js" defer=""></script></head><body><div id="__next"><header style="background:#fafafa"><nav><div class="MuiContainer-root MuiContainer-maxWidthLg mui-style-9wvnva"><div class="MuiStack-root mui-style-1xkh7h6"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-m89tgr">devth</a><div class="MuiStack-root mui-style-1x4jos1"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-v8mqvs">about</a><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-v8mqvs">overture</a><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-v8mqvs">glossary</a></div></div></div></nav></header><div class="MuiContainer-root MuiContainer-maxWidthLg mui-style-1hk2sn0"><div class="MuiGrid-root MuiGrid-container mui-style-1d3bbye"><div class="MuiGrid-root MuiGrid-item MuiGrid-grid-sm-1 mui-style-z689d8"></div><div class="MuiGrid-root MuiGrid-item MuiGrid-grid-xs-12 MuiGrid-grid-sm-10 mui-style-2mwbgk"><ul class="MuiTimeline-root MuiTimeline-positionRight mui-style-13a8w4s"><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2025</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">YAML was a bad idea for CI pipelines</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">4/14/2025</span> YAML was a bad idea for CI pipelines
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2024</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">You might not want a Monorepo</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">4/17/2024</span> Monorepos have become popular in recent years. What problems are they
solving, and what problems are they introducing?
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2023</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Will it survive entropy?</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">5/3/2023</span> For every component that makes up a modern web app engineers must decide
where on the spectrum of fully-managed to custom-built the solution should
lie. It can be useful to think through the lens of entropy and extropy.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2021</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Leave it better</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">4/22/2021</span> Great engineers improve everything they touch.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2020</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Essentiality of Open Source</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">4/7/2020</span> Open source is the foundation we build upon.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2019</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Notes on Unison
</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">9/17/2019</span> Notes on the Strange Loop talk 'Unison: a new distributed programming language'
by Paul Chiusano. Warning: mind expanding future tech 🤯
</span></h6></li><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Fast app generation and deployment</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">2/3/2019</span> The aim of this guide is to demonstrate the fastest way to create a client-side web app and get it deployed.</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2018</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">dec: Deep Environmental Config</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">7/10/2018</span> dec is a tiny library that embraces constraints to afford users equivalency
between multiple ways of setting environment variables according to their
particular requirements.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2017</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Compiled queries in Scala</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">2/26/2017</span> Compile a data structure representing a query into native
code to speed up a query loop using ASM.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2016</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Yetibot on Docker in 𝓧 minutes or less</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">1/6/2016</span> Yetibot is now on Docker!</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2015</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">A Simple type class</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">8/3/2015</span> Type classes provide a way of achieving ad hoc polymorphism</span></h6></li><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">ThrushCond is not a Monad</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">5/19/2015</span> Clojure has a useful macro called cond-> — let's explore a Scala equivalent</span></h6></li><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Monad laws in Scala</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">3/17/2015</span> The three Monad laws may seem pretty abstract at first, but they're quite
practical. Let's try to internalize the laws by running through two of Scala's
most popular monads and making sure they adhere.
</span></h6></li><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Beautiful constraint</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">1/30/2015</span> The freedom to do anything you wish is rarely a good thing, whether you're an
experienced engineer or a 2-year-old child.
</span></h6></li></ul></div></li><li class="MuiTimelineItem-root MuiTimelineItem-positionRight MuiTimelineItem-missingOppositeContent mui-style-628fjv"><div class="MuiTimelineSeparator-root mui-style-d04u73"><span class="MuiTimelineDot-root MuiTimelineDot-outlined MuiTimelineDot-outlinedPrimary mui-style-1txkj0z"></span><span class="MuiTimelineConnector-root mui-style-idv8vo"></span></div><div class="MuiTypography-root MuiTypography-body1 MuiTimelineContent-root MuiTimelineContent-positionRight mui-style-17gjue1"><h6 class="MuiTypography-root MuiTypography-h6 mui-style-1s6qh0k">2014</h6><ul style="list-style:none;padding:0"><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">On learning</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">10/5/2014</span> It wasn't until I was well into my twenties that I learned how to really learn</span></h6></li><li style="display:block;margin-top:25px;margin-bottom:25px"><h2 class="MuiTypography-root MuiTypography-h2 mui-style-yircub"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-1o57n5v">Corporate process should mirror open source</a></h2><h6 class="MuiTypography-root MuiTypography-subtitle1 mui-style-198olr7"><span style="font-size:1.2em;color:rgb(147, 147, 147)"><span style="color:red;font-family:monospace;font-size:0.7em;padding-right:0.3em">9/28/2014</span> Within any big tech corporation are many disparate teams. Most of these teams
know very little about the work, makeup or very existence of other teams, yet
they depend on each other's output, directly or indirectly.
</span></h6></li></ul></div></li></ul></div><div class="MuiGrid-root MuiGrid-item MuiGrid-grid-sm-1 mui-style-z689d8"></div></div></div><footer style="background-color:#fafafa;margin-bottom:0"><div class="MuiContainer-root MuiContainer-maxWidthLg mui-style-1v16n3n"><div class="MuiStack-root mui-style-1kjbk98"><div class="MuiStack-root mui-style-1x4jos1"><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-mpnz38">Home</a><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-mpnz38">About</a><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-mpnz38">Overture</a><a class="MuiTypography-root MuiTypography-inherit MuiLink-root MuiLink-underlineAlways mui-style-mpnz38">Glossary</a></div><p class="MuiTypography-root MuiTypography-body2 mui-style-cq9yas">© <!-- -->2025<!-- --> Trevor C. Hartman</p><p class="MuiTypography-root MuiTypography-body2 mui-style-1x1zu4f"><a style="text-transform:lowercase;font-style:italic;text-decoration:none;color:#999" href="https://github.com/devth/devth.com">view source</a> • <a style="text-transform:lowercase;font-style:italic;text-decoration:none;color:#999" href="https://en.wikipedia.org/wiki/Soli_Deo_gloria">Soli Deo Gloria</a></p></div></div></footer></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"posts":[{"content":"\nWithin any big tech corporation are many disparate teams. Most of these teams\nknow very little about the work, makeup or very existence of other teams, yet\nthey depend on each other's output, directly or indirectly.\n\nCommonly, when one team needs to use another's service or library, meetings are\nsetup, use cases are discussed, and all kinds of bureaucratic nonsense takes\nplace. Efficiency Failure.\n\nOpen source is need-driven. And while project authors and maintainers like to\nknow who their users are, it isn't a requirement. Code is built in a way to\nsolve a general problem. There is no need to \"discuss your use case\". A README\nprovides all the information a dev needs to:\n\n- Get the code\n- Run it\n- Generally understand what it does and how it works\n\nBeyond the README, well-built open source projects will provide docs, mailing\nlists, and an IRC channel. Everything you need to quickly ramp up or get help.\n\nContrast this with typical corporate projects:\n\n- missing or inadequatte README\n- no docs or docs that are several versions (or years) out-of-date\n- docs probably live on some separate wiki on the internal network and may or\n may not be referenced from project README\n- requests for help are met with meeting requests\n\nIt's no wonder corporate tech moves so slowly.\n\nA derivatory problem that corps suffer from is re-inventing the wheel over and\nover again instead of using existing solutions. That's because it's often easier\nto build something from scratch yourself than to try to deal with the awful\ninter-team politics and meetings-driven-communication. Over time this leads to a\nplethora of abandonware as teams move on and no longer need their\nhacked-together solution. Unnecessary LoC growth.\n\n## Solution\n\nStart developing your internal libraries and services as if they were open\nsource, even when you have 0 users. Write an excellent README and keep it\nup-to-date. Write in-repo docs to make it easier to keep them up-to-date, and be\ndisciplined about maintaining them. Write tests as necessary. Give examples.\nMake it so good that when people start using your code, you don't even know it\nbecause they didn't have any questions. But when they do have questions, **don't\nschedule a meeting**.\n","data":{"layout":"article","title":"Corporate process should mirror open source","categories":"process","comments":true,"excerpt":"Within any big tech corporation are many disparate teams. Most of these teams\nknow very little about the work, makeup or very existence of other teams, yet\nthey depend on each other's output, directly or indirectly.\n","image":{"feature":"open_source.jpg","teaser":"open_source_teaser.jpg","caption":"Boston Basin, WA. August, 2012"}},"dateString":"Sun Sep 28 2014","slug":"open-source-culture","filePath":"2014-09-28-open-source-culture.mdx"},{"content":"\nIt wasn't until I was well into my twenties that I learned how to really learn.\nAnd it was only by learning many hard concepts related to programming on my own\nthat I observed and proved what worked. It unlocked a sort of power. Realizing\nhow to learn nearly any hard topic is an incredibly useful skill. I found that\nefficient learning techniques had nothing to do with typical class structures\nfound in high school or university settings, which had previously drained my\ndesire to learn. In that regard, the subject of learning, or learning how to\nlearn is something I find very interesting and think about a fair amount, so\nwhen I found the [Learning How to\nLearn](https://class.coursera.org/learning-002) class on Coursera by Dr. Barbara\nOakley, Dr. Terrence Sejnowski I immediately signed up. Here are my notes and\nobservations gained from the class.\n\n## Modes\n\nThere are two primary modes of thought:\n\n0. **Focused**: this mode is engaged when you're doing something you already\n know how to do, quickly accessing the learned connections in your brain. The\n connections are very tightly spaced in the prefrontal cortex.\n\n0. **Diffuse**: this mode is for open minded thought that allows you to make new\n connections in your brain. You can think of this mode as exercising your\n brain muscle and letting your mind wonder. Edison and Dalí are famous\n examples of thinkers who practiced the diffuse mode: they would hold\n something in their hands while day-dreaming until dozing off to sleep at\n which point they'd drop the object and wake themselves back up in order to\n take what they learned and apply it in a focused mode. Diffuse mode relies on\n loosely spaced connections, and is engaged by drifting off to sleep and\n excercise.\n\n## Brain facts\n\n- The brain is the most complex device in the known universe\n- Acquiring skills takes years and years of practice (e.g. math, chess)\n- We are not consciously aware of how our brains work\n- There are 10\u003csup\u003e15\u003c/sup\u003e (a million billion) synapses in the brain\n- Brain connectivity is dynamic even after it matures\n- Our brains develop new synapses while we sleep\n\n## Techniques\n\n- **Pomodoro**: set a 25 minute timer to focus on a specific task or goal and reward\n yourself when you're done (e.g. read the Internet, get a coffee). Use this to\n build the neural patterns you need to learn new concepts. It helps to actually\n write your goal down. There are a few terminal and OS X apps that can help.\n Writing down the goal helps combat initial skatterbrain tendencies, which\n sometimes require continual re-focusing at first.\n\n- **Translation and rewriting**: when I read academic papers, I rarely\n understand the cryptic math syntax. A good way to break through is to\n translate the syntax to something you do understand. For me, this means\n Haskell. Then rewrite the plain text sections in your own words (tip: try\n writing a blog post that sumarizes the paper). You can't help but internalize\n the concepts when you do this.\n\n- **Handwritten notes**: when learning a library of any complexity, simply reading\n the source doesn't always help. You need to run it and observe the values in\n context and the execution path. A debugger can help here. While you're doing\n this, take handwriten notes on important functions and variable values. Sketch\n out an execution tree. You may never refer to these notes again, but the mere\n act of physically writing them makes abstract concepts more concrete.\n\n\n## Abstract Concepts\n\nAbstract concepts in particular require much more practice to create concrete\nneural thought patterns. The initial neural pattern is there but it is very\nweak. When you solve a problem again you strengthen that pattern. When you've\ndone it so many times you have it down, the thought pattern is very strong.\n\nStudy and focus intently, then take a break to allow the brain's diffuse mode\nto help you out with your conceptual understanding. \"Neural mortar has a chance\nto dry\".\n\n## Memory\n\n**Long term** memory vs **working memory**.\n\nWorking memory (prefrontal cortex) is what you're currently focusing and working\non. It can hold about 4 chunks of information. We naturally group things into\nchunks (e.g. phone numbers). Working memory is like a blackboard.\n\nLong term memory is like a storage warehouse, spread over different areas of the\nbrain. To store something in long term memory you need to revisit it several\ntimes. We can store billions of items, but they can bury each other. Spaced\nrepetition is one technique of storing something in long term memory: repeat\nsomething over a number of days.\n\n## Sleeping\n\nBeing awake causes toxic products in your brain. When you sleep your brain cells\nshrink, causing an increased space between them, allowing fluid to wash the\ntoxins out.\n\nIt's also an important part of the memory and learning process: it organizes the\nthings you're learning. It rehearses neural patterns of things you're learning,\nstrengthening and deepening them.\n\nIf you focus on what you're learning right before you take a nap you have an\nincreased chance of dreaming about it, especially if you want to dream about it.\nDreaming about what you're studying can enhance your ability to understand: it\nconsolidates your memories into easier-to-grasp chunks.\n\n## Activity\n\nPhysical excercise engages the diffuse mode of thinking. This is why it's so\nimportant to excercise. I try to make some form of excercise a part of my daily\nroutine as a lifestyle rather than a discipline. For example, I used to commute\nby bike 5 miles each way to work when I lived in Seattle. This was an excellent\nway to start the morning, and end the work day. Now that I work from home, I\nmountain bike over lunch during the summer, and walk, run or go to the climbing\ngym during the winter. Many times as I've thought about a problem while in the\nzen state of singletrack mountain biking, the diffuse mode of thinking has\nbrought a very different perspective and creativity to problem solving.\n\n## Environment\n\nIt's important to work in a space that inspires you to create. If you work from\nhome, carve out a dedicated space to work, and spend time desigining or\nremodeling if necessary to make it suit your needs. For me, this means a modern\nspace with interesting art and a mix of home made physical computing hackery.\n\nIt also helps if you're working near other people are creating and hacking. If\nyou work from home, try to make time to work from coffee shops, find a coworking\nspace to occassionally work from, or even meet up with a friend who freelances\nto work together.\n\n## Chunking\n\nA chunk is a network of neurons working together, built from practice and\nrepetition. New concepts often don't make sense in relation to other concepts as\npart of the bigger picture. It's important to connect new material to existing\nknowledge. Chunks are pieces of information that are bound together. One of\nthe first steps to learning is creating chunks which are connected to knowledge\nthrough meaning. Once you chunk an idea you don't need to remember all the\nunderlying details.\n\n### How to form a chunk\n\nConsider learning a new song on the guitar. The entire song could be considered\none large chunk, but can also be broken down into mini-chunks that are easier to\nlearn through repetition. You can then join the mini-chunks together into larger\nchunks. The joining process can be continually repeated to build up larger and\nlarger chunks. The point of joining small chunks into a larger one is so that\nyou don't even need to consciously think about connecting the individual neural\nnetworks.\n\nLearning a language is another example. At first you learn pronounciation and\nhow to string together words to form simple sentences. As you progress, you can\nmore creatively string together complex phrases to communicate subtleties.\n\nWhen learning a mental concept, form a chunk by:\n\n0. Focus your undivided attention on the subject\n0. Understand the basic idea you're trying to chunk: let focus and diffuse mode\n alternate in order to synthesize the gist. There are two approaches that work\n together. The bottom up approach involves chunking and repetition while top\n down involves understanding the big picture. Both perspectives are vital and\n meet at the context level, which helps you learn how and when a chunk applies\n to the big picture.\n0. Practice and repetition\n\nIt helps to learn major concepts first, then go back in and fill in the details.\n\n\n## Further reading\n\n- [Learning How to Learn](https://www.coursera.org/learn/learning-how-to-learn)\n — the class this post references, now available \"on dmeand\"\n- [Brains Sweep Themselves Clean Of Toxins During Sleep](http://www.npr.org/sections/health-shots/2013/10/18/236211811/brains-sweep-themselves-clean-of-toxins-during-sleep)\n — NPR\n","data":{"layout":"article","title":"On learning","categories":"learning","comments":true,"excerpt":"It wasn't until I was well into my twenties that I learned how to really learn","toc":true,"image":{"feature":"on_learning_1024_256.jpg","teaser":"on_learning_410_228.jpg","caption":"Highline Singletrack, Rimrocks, Billings, MT. October 2014"}},"dateString":"Sun Oct 5 2014","slug":"on-learning","filePath":"2014-10-05-on-learning.mdx"},{"content":"\nThe freedom to do anything you wish is rarely a good thing, whether you're an\nexperienced engineer or a 2-year-old child. Our interests and desires are too\ndiverse and of varying merit to actually produce anything good without\nconstraint. That's not to say freedom is intrinsically bad: sometimes we are\nable to choose the right constraints ourselves; other times they are chosen for\nus. Either way, constraint is what guides and sometimes dictates, removes\ndecisions and forces creative thinking. Constraint is a framework.\n\nThese are some constraints I've embraced over my years as a developer.\n\n## Terminal\n\nThe terminal is a beautiful constraint. Because of its spartan nature, tools\nbuilt in terminal turn out more consistent in behavior and UX, faster, and less\nbug-ridden than their graphical counterparts. You know right away when an app\nwas built on Java/Swing vs native APIs. Look at all differences between UI junk\nin Chrome vs Safari on OS X. Then compare that to the simulated UI junk in Flash\nor Silverlight. Then look at Eclipse. It's grotesque and highly inconsistent.\n\nNow compare operating systems. The way in which humans interact with a computer\nshould be more of a UX science and less of an artifact of personalization, and\nespecially not the whim of corporations whose interests (profit) don't\nnecessarily align with advancing human-computer interaction.\n\nYou may say terminal is rudimentary. Well I agree, but that's part of what makes\nit so great. In my early days, it annoyed me greatly when I had to resort to\nusing the terminal, but I eventually embraced it, moving all my tools and\nworkflow into it because it's consistent, fast, reliable, and most importantly\n_automation is intrinsic_.\n\n## Keyboard\n\nAs many have observed, using a mouse is almost always less efficient than using\na keyboard shortcut or command (there are exceptions, but not many apply to the\ntype of work done by developers). I've increasing eschewed mouse use and it\nso happens that Terminal and vim are a very natural way to do so. The reliance\non keyboard shortcuts is a productivity framework.\n\n## $EDITOR\n\nConsistent with my use of terminal and keyboard shortcuts, I use vim, though\nEmacs is also an excellent choice for the precise and thoughtful\nautomation-and-productivity-minded developer. Many of the points regarding\nTerminal's simplicity and consistency also apply to these great editors,\noriginally built by yesterday's technologists and finely tuned over the decades.\n`$EDITOR` only gets better with age. Vim's minimalism and\nsimplicity give way to an uncluttered mind, allowing you to focus on the problem\nat hand rather than the tools—a framework for clear thinking.\n\n\u003cimg src=\"/images/vim_emacs.png\" alt=\"Vim and Emacs vs Atom\" /\u003e\n\n## Functional Programming\n\nIn a similar way that tools constrain us, the functional style of programming\nintroduces tremendous restrictions on how we write programs. FP is a huge topic\nthat I won't even attempt to cover here, but we can briefly take a look at some\nof its tenets and the benefits that come with.\n\n### Composition\n\nOne way to constrain your code is to implement functions purely in terms of core\nor library functions, as in this JavaScript example.\n\n```javascript\nconst mergeValues = _.compose(_.partialRight(_.reduce, _.merge), _.values);\nmergeValues({ a: { foo: 1, bar: 2 }, b: { qux: 3 } });\n//=\u003e {foo: 1, bar: 2, qux: 3}\n```\n\nI'm specifically avoiding the use of `function` in the definition of\n`mergeValues`. That is a strong indicator to the reader that _nothing_ fancy is\ngoing on here. It's simply a composition of library functions (lodash.js) which\nprevents me from writing any custom code. Instead, I'm forced to think of\nexisting functions, and compose them in a way that produces intended behavior.\n\nComposition is the foundation of reusability — a worthy pursuit.\n\n### Immutability\n\nThe benefits of immutability are quite well-known by now, even among OO\nprogrammers. It prevents all sorts of nonsense like defensive copying and\ncombinatorial explosion of possible states and aligns well with reality and\nintuition. After all, a value is a value. When you say `2 + 3` you are not\nmutating `2`; you're producing a new, immutable, unchangeable value, `5`. Why\nshould `2 :: List(3)` behave any differently?\n\nImmutability enforces a restriction on how we write algorithms. No longer can we\nloop over some collection and store computed results on another collection via\nmutation. Instead we look to elegant functional constructs like `map` and\n`fold`.\n\nSee these excellent presentations by Rich Hickey if you'd like to go deeper on\nthe benefits of immutability. It's a wonderful constraint.\n\n- [The Value of Values](http://www.infoq.com/presentations/Value-Values)\n- [Are We There Yet?](http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey)\n\n### Referential transparency\n\nReferential transparency (RT) is an attribute of pure functions, which do not\nperform side effects (e.g. IO, mutation), mutate their arguments, nor depend on\noutside variables. RT says you can substitute the value for any expression.\n\n```scala\nval double = {x: Int =\u003e x * 2}\ndouble(4)\n// =\u003e 8\n// double(4) simply represents the expression 4 * 2 so let's substitute it:\n4 * 2\n// =\u003e 8\n// 4 * 2 represents 8 so let's substitute it:\n8\n//=\u003e 8\n```\n\nIt's obvious how the substitution works in this context. Now imagine the use of\n`double` in a slightly wider context:\n\n```scala\ndouble(50) / double(5)\n//=\u003e 100 / 10\n//=\u003e 10\n```\n\nRunning this code in your head is very easy because you can safely substitute\n`100` and `10`. This concept of substitution scales up as your context gets\nwider and wider, as it naturally does in a program of any meaningful size. This\nis called equational reasoning. RT gives you the ability to reason locally in a\nsmall context instead of having to worry about who is mutating what or where\nunpredictable values are coming from. It also makes functions more testable,\ncomposable, and parallelizable.\n\n## Constraint as freedom\n\nBeing intentional about which constraints you adopt and embrace is freeing. No\nlonger are you lost at sea, being tossed about. Your constraints are fixed, and\nyou have the freedom to become highly skilled at working with and around them,\nusing them as a rudder to guide you during the creative problem-solving process.\n\nMaker, choose your constraints wisely.\n\n## Further exploration\n\n- [Constraints Liberate, Liberties Constrain](https://www.youtube.com/watch?v=GqmsQeSzMdw):\n an excellent talk by the always-brilliant [Rúnar Bjarnason](http://blog.higher-order.com/),\n co-author of [Functional Programming in Scala](https://www.manning.com/books/functional-programming-in-scala)\n","data":{"layout":"article","title":"Beautiful constraint","categories":"philosophy","comments":true,"excerpt":"The freedom to do anything you wish is rarely a good thing, whether you're an\nexperienced engineer or a 2-year-old child.\n","image":{"feature":"constraint_1200_300.jpg","teaser":"constraint_410_228.jpg","caption":"Oak Alley Plantation, Vacherie, LA. September 2014"}},"dateString":"Fri Jan 30 2015","slug":"beautiful-constraint","filePath":"2015-01-30-beautiful-constraint.mdx"},{"content":"\nThe three Monad laws may seem pretty abstract at first, but they're quite\npractical. Let's try to internalize the laws by running through two of Scala's\nmost popular monads and making sure they adhere. We could use something like\n[ScalaCheck](http://scalacheck.org/) to more rigorously check these laws, but\nthe purpose of this post is to help internalize them, so we'll only be manually\nverifying them using our intuition.\n\nHere are the laws, from [Monad laws](https://wiki.haskell.org/Monad_laws) on HaskellWiki.\n\n1. Left identity: `return a \u003e\u003e= f ≡ f a`\n1. Right identity: `m \u003e\u003e= return ≡ m`\n1. Associativity: `(m \u003e\u003e= f) \u003e\u003e= g ≡ m \u003e\u003e= (\\x -\u003e f x \u003e\u003e= g)`\n\nIn Haskell, `return` is used to \"inject a value into the\nmonadic type\". In Scala, we do this via constructors (unless you're using\nScalaz, in which case you probably already know everything this post has to\noffer).\n\nThe `\u003e\u003e=` operator in Haskell corresponds to Scala's\n`flatMap` method.\n\n## List\n\nThe List Monad deals with the context of non-determinism—that is, it represents\nmultiple values. When we run multiple lists through a sequence comprehension we\nend up with the all combinations of values from each list.\n\n```scala\nfor (x \u003c- List(1, 2, 3); y \u003c- List('a', 'b', 'c')) yield (x, y)\n=\u003e List((1,a), (1,b), (1,c), (2,a), (2,b), (2,c), (3,a), (3,b), (3,c))\n```\n\nNow the laws. Let's setup two simple functions `f` and `g`, both of type `Int =\u003e\nList[Int]`.\n\n```scala\n// Let f be a function that takes an Int and produces a List of its\n// neighboring Ints along with itself:\nval f: (Int =\u003e List[Int]) = x =\u003e List(x - 1, x, x + 1)\n\n// Let g be a function that takes an Int x\n// and produces a List containing +x and -x\nval g: (Int =\u003e List[Int]) = x =\u003e List(x, -x)\n```\n\n### Left identity\n\n```scala\nval a = 2\nval lhs = List(a).flatMap(f)\n=\u003e List(1, 2, 3)\n\nval rhs = f(a)\n=\u003e List(1, 2, 3)\n\nlhs == rhs\n=\u003e true\n```\n\n### Right identity\n\n```scala\nval m = List(2)\n\nval lhs = m.flatMap(List(_))\n=\u003e List(2)\n\nval rhs = m\n=\u003e List(2)\n\nlhs == rhs\n=\u003e true\n```\n\n### Associativity\n\n```scala\nval m = List(1, 2)\n\nval lhs = m.flatMap(f).flatMap(g)\n=\u003e List(0, 0, 1, -1, 2, -2, 1, -1, 2, -2, 3, -3)\n// Sidenote: now do you see what is meant by non-determinism?\n\nval rhs = m.flatMap(x =\u003e f(x).flatMap(g))\n=\u003e List(0, 0, 1, -1, 2, -2, 1, -1, 2, -2, 3, -3)\n\nlhs == rhs\n=\u003e true\n```\n\nLooks good to me.\n\n## Option\n\nLet's create new test functions `f` and `g` of type `Int =\u003e\nOption[Int]`. Given the type signature, it's natural to think\nof `f` and `g` as _partial functions_ that are only defined on certain inputs.\n\n```scala\n// If x is not less than 10, return 2x\nval f: (Int =\u003e Option[Int]) = x =\u003e if (x \u003c 10) None else Some(x * 2)\n\n// If x is reater than 50, return x + 1\nval g: (Int =\u003e Option[Int]) = x =\u003e if (x \u003e 50) Some(x + 1) else None\n```\n\nFor the sake of testing our laws, the implementations of these functions really\ndon't matter as long as the types line up.\n\n### Left identity\n\n```scala\nval a = 30\nval lhs = Option(a).flatMap(f)\n=\u003e Some(60)\n\nval rhs = f(a)\n=\u003e Some(60)\n\nlhs == rhs\n=\u003e true\n```\n\n### Right identity\n\n```scala\nval m = Option(30)\n\nval lhs = m.flatMap(Option(_))\n=\u003e Some(30)\n\nval rhs = m\n=\u003e Some(30)\n\nlhs == rhs\n=\u003e true\n```\n\n### Associativity\n\n```scala\nval m = Option(30)\n\nval lhs = m.flatMap(f).flatMap(g)\n=\u003e Some(61)\n\nval rhs = m.flatMap(x =\u003e f(x).flatMap(g))\n=\u003e Some(61)\n\nlhs == rhs\n=\u003e true\n```\n\n## The end\n\nI hope this post helped you internalize the Monad laws. If you need more\npractice, continue this exercise in your REPL for the `Try`\nand `Either` monads, or better yet: create your own Monad and\nverify that it obeys the laws!\n\n### Further reading\n\n- [Monad laws — learning Scalaz](http://eed3si9n.com/learning-scalaz/Monad+laws.html)\n- [Monad laws — HaskellWiki](https://wiki.haskell.org/Monad_laws)\n- [A Fistful of Monads — Learn You a Haskell](http://learnyouahaskell.com/a-fistful-of-monads#monad-laws)\n","data":{"layout":"article","title":"Monad laws in Scala","categories":"scala","comments":true,"excerpt":"The three Monad laws may seem pretty abstract at first, but they're quite\npractical. Let's try to internalize the laws by running through two of Scala's\nmost popular monads and making sure they adhere.\n","image":{"feature":"monad_laws_1200_300.jpg","teaser":"monad_laws_410_228.jpg","caption":"Mount St. Helens from Mount Adams, WA. July 2011"}},"dateString":"Tue Mar 17 2015","slug":"monad-laws-in-scala","filePath":"2015-03-17-monad-laws-in-scala.mdx"},{"content":"\nClojure has a useful macro called\n[cond-\u003e](https://clojuredocs.org/clojure.core/cond-%3E) that conditionally\nthreads an initial value through a series of predicate/function pairs only\napplying each function if its predicate returns true. In this post we're going\nto look at a Scala representation, and whether it fits the shape and laws of any\n[common algebraic structures](https://en.wikipedia.org/wiki/Outline_of_algebraic_structures#Types_of_algebraic_structures).\nWe'll look at Functor, Monad, Semigroup, and Monoid.\n\n**TL;DR** — [view the full code listing](https://gist.github.com/devth/308da859e5b584340f93).\n\nLet's start with an example in Clojure. We want to build up a request based on\nsome arbitrary conditions:\n\n```clojure\n;; sample values from user input\n\n(def user-id 1)\n(def user-name \"devth\")\n(def user-address nil)\n(def accept :json)\n\n;; validation and helper functions\n\n(def accept-map )\n(defn valid-accept? [a] (accept-map a))\n(defn set-header [req k v] (update-in req [:headers] assoc k v))\n(defn set-param [req k v] (update-in req [:params] assoc k v))\n\n;; build up a request map using cond-\u003e to decide which items to add to params\n;; and headers maps\n\n(def request\n (cond-\u003e\n user-name (set-param :user-name user-name)\n user-address (set-param :user-address user-address)\n (valid-accept? accept) (set-header :accept accept)))\n\n;; request value:\n\n{:target \"/users\"\n :query-params\n :headers\n```\n\nSince Clojure's `-\u003e` operator is sometimes referred to as\nthe \"thrush\" operator, I'm going to call `cond-\u003e`\nin Scala `ThrushCond`.\n\nFirst let's model the `Request` and helpers equivalent to\nthose we used in the Clojure example:\n\n```scala\ncase class Request(\n target: String,\n params: Map[String, String] = Map.empty,\n headers: Map[String, String] = Map.empty) {\n\n // validation and helper functions\n val acceptMap = Map(\"html\" -\u003e \"text/html\", \"json\" -\u003e \"application/json\")\n val isValidAccept: (String =\u003e Boolean) = acceptMap.isDefinedAt _\n\n def addParam(k: String, v: String) = this.copy(params=params.updated(k, v))\n def addHeader(k: String, v: String) = this.copy(headers=headers.updated(k, v))\n}\n\n// sample values from user input\nval userId: Int = 1\nval userName: Option[String] = Some(\"devth\")\nval address: Option[String] = None\nval accept = \"json\"\n```\n\nNow we'll create the `ThrushCond` class that takes any number\nof predicate/function pairs, provides a `guard` function to only run a function\nif the predicate passes, a method to flatten the chain of functions via\ncomposition, and finally a `run` method that takes a value and\nruns it through the chain.\n\n```scala\ntype Step[A] = (A =\u003e Boolean, A =\u003e A)\n\ncase class ThrushCond[A](steps: Step[A]*) {\n /** Perform a pipeline step only if the value meets a predicate */\n def guard[A](pred: (A =\u003e Boolean), fn: (A =\u003e A)): (A =\u003e A) =\n (a: A) =\u003e if (pred(a)) fn(a) else a\n /** Compose the steps into a single function */\n def comp = Function.chain(steps.map { step =\u003e guard(step._1, step._2) })\n /** Run a value through the pipeline */\n def run(a: A) = comp(a)\n}\n```\n\nTry it out:\n\n```scala\nval requestPipeline = ThrushCond[Request](\n ({_ =\u003e userName.isDefined}, {_.addParam(\"userName\", userName.get)}),\n ({_ =\u003e address.isDefined}, {_.addParam(\"address\", address.get)}),\n ({_.isValidAccept(accept)}, {r =\u003e r.addHeader(\"accept\", r.acceptMap(accept))}))\n\nval request = requestPipeline run Request(\"/users\")\n\n//=\u003e\nRequest(/users,Map(userName -\u003e devth),Map(accept -\u003e application/json))\n```\n\nAs you can see, it correctly skipped the 2nd step based on the\n`address.isDefined` condition and runs the other steps because\ntheir predicates evaluate to `true`.\n\nWill this work as one of the algebraic structures mentioned at the start?\n\n## Functor\n\nConsider Functor's `fmap`:\n\n```scala\ndef fmap[A, B](f: A =\u003e B): F[A] =\u003e F[B]\n```\n\nIn our case, both `A` are the\nsame type, `Request` produces a\nfunction that fits, but we could easily use that with an existing Functor,\ne.g.:\n\n```scala\nval step: Request =\u003e Request =\n guard({_ =\u003e userName.isDefined}, {setParam(_, \"userName\", userName.get)})\nSome(Request(\"/users\")).map(step)\n```\n\nThe essense of `ThrushCond`\nitself so it makes no sense to design a new Functor around it.\n\n## Monad\n\nLikewise, Monad's `flatMap`:\n\n```scala\ndef flatMap[A, B](f: A =\u003e F[B]): F[A] =\u003e F[B]\n```\n\nWe could make `guard`'s\nsignature, but there's no point in doing so for the same reason it didn't make\nsense for Functor: the essense is not how a transformation is applied, it's\n_whether_ the transformation is applied, and because of the signature, the\ndecision whether to perform a transformation must be embedded in the\ntransformation itself, hence `guard`.\n\n## Semigroup\n\nLet's see if it meets Semigroup's associativity laws:\n\n```scala\ncase class F(x: Int)\nval f = F(10)\nval always = Function.const(true) _\n\nval mult2: F =\u003e F = guard(always, {f =\u003e f.copy(x = f.x * 2)})\nval sub4: F =\u003e F = guard(always, {f =\u003e f.copy(x = f.x - 4)})\nval sub6: F =\u003e F = guard(always, {f =\u003e f.copy(x = f.x - 6)})\n\nval g: (F =\u003e F) = (mult2 andThen sub6) andThen sub4\nval h: (F =\u003e F) = mult2 andThen (sub6 andThen sub4)\n\ng(f)\n//=\u003e F(10)\nh(f)\n//=\u003e F(10)\n```\n\n`guard` is associative when composed with itself because\n[function composition is associative](https://en.wikipedia.org/wiki/Function_composition#Properties).\nBecause of this associative binary operation we can provide evidence that\n`ThrushCond`'s\n`Semigroup` representation:\n\n```scala\nimport scalaz._, Scalaz._\n\ncase object ThrushCond {\n /** Evidence of a Semigroup */\n implicit def thrushCondSemigroup[A]: Semigroup[ThrushCond[A]] =\n new Semigroup[ThrushCond[A]] {\n def append(t1: ThrushCond[A], t2: =\u003e ThrushCond[A]): ThrushCond[A] =\n ThrushCond[A]((Function.const(true), t2.comp compose t1.comp))\n }\n}\n```\n\nWe've defined a Semigroup over the set of all\n`ThrushCond[A]`s. What does this give us? We can now combine\nany number of `ThrushCond`s using Semigroup's\n`|+|` operator. A simple example using\n`ThrushCond[Int]`:\n\n```scala\nimport ThrushCond.thrushCondSemigroup\n\nval addPipeline = ThrushCond[Int](\n ((_ \u003e 10), (_ + 2)),\n ((_ \u003c 20), (_ + 20)))\n\nval multPipeline = ThrushCond[Int](\n ((_ == 70), (_ * 10)),\n ((_ \u003e 0), (_ * 7)))\n\nval pipeline = addPipeline |+| multPipeline\n\n// Examples\nmultPipeline run 70 //=\u003e 70 * 10 * 7 == 4900\npipeline run 2 //=\u003e (2 + 20) * 7 == 154\npipeline run 12 //=\u003e (12 + 2 + 20) * 7 == 238\n```\n\n## Monoid via PlusEmpty\n\nMonoids are Semigroups with an identity element.\n`ThrushCond`'s identity is simply a\n`ThrushCond` arguments.\nHowever, as [@lmm mentioned in the\ncomments](http://devth.com/2015/thrush-cond-is-not-a-monad/#comment-2082941866):\n\n\u003e it's not ThrushCond itself that forms a Monoid but rather ThrushCond[A] for\n\u003e any given A\n\nThis is where `PlusEmpty` comes in.\n`PlusEmpty` is a [\"universally quantified\nMonoid\"](https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/main/scala/scalaz/PlusEmpty.scala#L3-7)\nwhich means it's like a Monoid but for first-order `* -\u003e *`\ntypes instead of proper `*` types.\n`PlusEmpty` itself is a higher-order `(* -\u003e *) -\u003e *` type. A\nhelpful quote from #scalaz:\n\n\u003e tpolecat: so `String` is a monoid, but\n\u003e `List` (which means that\n\u003e `List[A]`)\n\nTo provide evidence of a `PlusEmpty`, we must be able to implement these two\nmethods (where `F`):\n\n```scala\ndef plus[A](a: F[A], b: =\u003e F[A]): F[A] // from Plus\ndef empty[A]: F[A] // from PlusEmpty which extends Plus\n```\n\nWe already implemented `plus` for `Semigroup`'s `append`, and\n`empty` is simply a `ThrushCond` without args.\n\n```scala\ncase object ThrushCond {\n /** Evidence of a PlusEmpty */\n implicit def thrushCondPlusEmpty: PlusEmpty[ThrushCond] =\n new PlusEmpty[ThrushCond] {\n def plus[A](a: ThrushCond[A], b: =\u003e ThrushCond[A]): ThrushCond[A] =\n ThrushCond[A((Function.const(true), b.comp compose a.comp)))\n\n def empty[A]: ThrushCond[A] = ThrushCond[A]()\n }\n /** Use PlusEmpty to provide evidence of a Monoid[Request] */\n implicit def requestMonoid: Monoid[ThrushCond[Request]] =\n thrushCondPlusEmpty.monoid[Request]\n /** Evidence of a Semigroup */\n implicit def thrushCondSemigroup[A]: Semigroup[ThrushCond[A]] =\n new Semigroup[ThrushCond[A]] {\n def append(t1: ThrushCond[A], t2: =\u003e ThrushCond[A]): ThrushCond[A] =\n ThrushCond[A]((Function.const(true), t2.comp compose t1.comp))\n }\n}\n```\n\nLet's go back to our `Request` example in Clojure and use\nPlusEmpty's `\u003c+\u003e` to combine separate transformation\npipelines:\n\n```scala\nimport ThrushCond._ // evidence\n\nval userPipeline = ThrushCond[Request](\n ({_ =\u003e userName.isDefined}, {_.addParam(\"userName\", userName.get)}),\n ({_ =\u003e address.isDefined}, {_.addParam(\"address\", address.get)}))\n\nval headerPipeline = ThrushCond[Request](\n ({_.isValidAccept(accept)}, {req =\u003e\n req.addHeader(\"accept\", req.acceptMap(accept))}))\n\n// \u003c+\u003e is an alias for plus\nval requestPipeline = userPipeline \u003c+\u003e headerPipeline\n// A PlusEmpty[ThrushCond] is implicitly obtained and used to plus the two\n// ThrushCond[Request]s\n\nrequestPipeline run Request(\"/users\")\n//=\u003e\nRequest(/users,Map(userName -\u003e devth),Map(accept -\u003e application/json))\n```\n\nBecause `PlusEmpty` can derive a Monoid for a given type, we can combine any\nnumber of `ThrushCond`s from a List. Let's construct one more\n`ThrushCond` pipeline that conditionally adds a cache-control\nheader and try out our Monoid using `Foldable`'s\n`suml`:\n\n```scala\nimport scala.language.postfixOps\n\nval shouldCache = false\n\nval cachePipeline = ThrushCond[Request](\n ({_ =\u003e !shouldCache}, {_.addHeader(\"cache-control\", \"no-cache\")}))\n\nval requestPipeline = List(userPipeline, headerPipeline, cachePipeline) suml\nrequestPipeline run Request(\"/users\")\n//=\u003e\nRequest(/users,\n Map(userName -\u003e devth),\n Map(accept -\u003e application/json, cache-control -\u003e no-cache))\n```\n\nThrushCond is not a Monad, nor a Functor, **but it is a PlusEmpty from which can\nbe derived a Monoid**.\n\n[View the full code listing](https://gist.github.com/devth/308da859e5b584340f93).\n\n_Updated July 1, 2015: incorporated lmm's PlusEmpty suggestion._\n","data":{"title":"ThrushCond is not a Monad","categories":"scala","comments":true,"excerpt":"Clojure has a useful macro called cond-\u003e — let's explore a Scala equivalent"},"dateString":"Tue May 19 2015","slug":"thrush-cond-is-not-a-monad","filePath":"2015-05-19-thrush-cond-is-not-a-monad.mdx"},{"content":"\nType classes provide a way of achieving [ad hoc\npolymorphism](https://en.wikipedia.org/wiki/Ad_hoc_polymorphism). We'll look at\nwhat they are, why they're useful, how they're typically encoded in Scala.\nWe'll make our own `Simple` type class that has a single\noperation called `simplify` returning a\n`String`, and provide instances of it for Scala's\n`List` and `Int` classes, along with our own\nclass `C`. Finally, we'll add convenient syntax in the Scalaz\nstyle to make it extremely easy to use the `simplify`\noperation. This might be the most hands-on, easy to understand explanation of\ntype classes you've ever encountered!\n\n## Rationale\n\nA common way to express polymorphism in Scala is inheritance. This is brittle\nbecause the polymorphic relationship between supertype and subtype must be\ndefined where the subtype itself is defined. For example, if you have an\n`Animal` trait, your `Dog` class must\ndeclare its relationship as a subtype of `Animal` as part of\nits definition. This is problematic when you do not own potential subtypes (i.e.\nthey're provided by a library). It also tightly couples the subtype to all of\nits implementations of its potential supertypes. A given subtype may already\nhave all the methods needed to implement the behavior of a given supertype but\ndoesn't know or care about the supertype at definition time. Though less common,\nanother problem is a subtype may be able to implement a supertype's operations\nin multiple, distinct ways (e.g. multiplication and addition implementations of\na Monoid over `Int`). With inheritance this is impossible.\n\n## Scalaz\n\nLet's start with one of the simplest type classes in Scalaz:\n`Equal`. `Equal` describes \"a type safe\nalternative to universal equality\" (if you're wondering why this is useful or\nnecessary, ask me in the comments). Here's an example of providing an instance\nof `Equal` over our own class `C` that uses\nvalue equality to compare two instances.\n\n```scala\nclass C(val name: String)\n\nimplicit val equalC: Equal[C] = new Equal[C] {\n override def equal(c1: C, c2: C) = c1.name == c2.name\n}\n```\n\nNow let's say we want a `notEqual` method that works on any\n`A` provided there is evidence of\n`Equal[A]`:\n\n```scala\ndef notEqual[A: Equal](a1: A, a2: A): Boolean =\n !implicitly[Equal[A]].equal(a1, a2)\n\nval c1 = new C(\"foo\")\nval c2 = new C(\"bar\")\n\nnotEqual(c1, c2)\n//=\u003e true\n```\n\nSidenote for those unfamiliar with _context bounds_: the context bound `[A:\nEqual]` is what ensures we have an implicit\n`Equal[C]` instance available when running `notEqual(c1,\nc2)`. An equivalent and perhaps more clear implementation\nwithout context bounds would look like this:\n\n```scala\ndef notEqual[A](a1: A, a2: A)(implicit e: Equal[A]) = !e.equal(a1, a2)\nnotEqual(c1, c2)\n//=\u003e true\n```\n\nHowever, there's an even more concise way of writing this using context bounds\nand Scalaz' `/==` operator for instances of\n`Equal`, or even its unicode `≠` alias:\n\n```scala\ndef notEqual[A: Equal](a1: A, a2: A): Boolean = a1 /== a2\n// or\ndef notEqual[A: Equal](a1: A, a2: A): Boolean = a1 ≠ a2\n```\n\nSince Scalaz already provides these operators, `notEqual` is purely\ndidactic.\n\nAnother common typelcass in Scalaz is `Show`. This corresponds\nto Haskell's `Show` type class, and is used to indicate a\ntype that can be represented in some way as a string, e.g. for logging or\nprinting in a REPL. Here's an instance for our `C` class\nexample from before.\n\n```scala\nimplicit val showC: Show[C] = new Show[C] {\n override def show(c: C) = s\"C[name=${c.name}]\"\n}\n```\n\nScalaz provides syntax helpers that allow us to simply call\n`.show` on any type that provides evidence of\n`Show`. We'll look at how that works in detail toward the end.\n\n```scala\nnew C(\"qux\").show\n//=\u003e res22: scalaz.Cord = C[name=qux]\n```\n\n## Simple\n\nNow that we've gotten a taste for using a few of Scalaz' type classes, let's\nbuild our own, along with some syntax helpers in the Scalaz style.\n\n```scala\ntrait Simple[F] {\n def simplify(f: F): String\n}\n```\n\nThis is our type class definition. It defines a single operation\n`simplify` for a given `F` and returns a\n`String`. Before we provide instances, let's define a method\nthat expects a `Seq` of `Simple` instances\nand outputs them separated by newlines.\n\n```scala\ndef manySimple[A](simples: Seq[A])(implicit s: Simple[A]): String =\n \"Many simples:\\n\\t\" + simples.map(s.simplify).mkString(\"\\n\\t\")\n```\n\nWe can use this to easily try out instances. Let's start with an instance of\n`C`:\n\n```scala\nimplicit def simpleC: Simple[C] = new Simple[C] {\n override def simplify(c: C) = s\"Simplified: ${c.show}\"\n}\n```\n\nWe can manually call this by implicitly obtaining a\n`Simple[C]` then calling `simplify`:\n\n```scala\nimplicitly[Simple[C]].simplify(new C(\"hello\"))\n```\n\nOr we can try out the `manySimple` method:\n\n```scala\nmanySimple(Stream(new C(\"foo\"), new C(\"bar\"), new C(\"qux\")))\n//=\u003e Many simples:\n//=\u003e Simplified: C[name=foo]\n//=\u003e Simplified: C[name=bar]\n//=\u003e Simplified: C[name=qux]\n```\n\nLet's try another one: `Int`.\n\n```scala\nimplicit val simpleInt: Simple[Int] = new Simple[Int] {\n override def simplify(i: Int) = s\"Simplified Int with value of $i\"\n}\n```\n\nThis is getting easy, right?\n\n```scala\nimplicitly[Simple[Int]].simplify(123)\n//=\u003e Simplified Int with value of 123\n```\n\nWe can even provide an instance for `List[A]` but only if\n`A` itself has a `Simple` instance:\n\n```scala\n// Evidence for Simple[List[A]] given evidence of Simple[A]\nimplicit def simpleList[A: Simple]: Simple[List[A]] = new Simple[List[A]] {\n override def simplify(l: List[A]) = {\n val simplifyA = implicitly[Simple[A]].simplify _\n s\"Simplified list:\\n${l.map(simplifyA).mkString(\"\\n\")}\"\n }\n}\n```\n\nTry it out:\n\n```scala\nimplicitly[Simple[List[Int]]].simplify((1 to 5).toList)\n//=\u003e Simplified list:\n//=\u003e Simplified Int with value of 1\n//=\u003e Simplified Int with value of 2\n//=\u003e Simplified Int with value of 3\n//=\u003e Simplified Int with value of 4\n//=\u003e Simplified Int with value of 5\n```\n\nHopefully you have a feel for how this works (ignoring how unuseful our\n`Simple` is IRL). Next, let's follow Scalaz lead and\nprovide some convenient syntax for working with our new type class; typing\n`implicitly[Simple[_]]` over and over again is starting to get\nold.\n\n## Syntax\n\nWouldn't it be nice if we could just call `.simplify` on objects which provide\nevidence of a `Simple`? Well, we can via some neat implicit\ntricks. Check it out:\n\n```scala\nfinal class SimpleOps[F](val self: F)(implicit val F: Simple[F]) {\n final def /^ = F.simplify(self)\n final def ⬈ = F.simplify(self)\n}\n\ntrait ToSimpleOps {\n implicit def ToSimpleOps[F](v: F)(implicit F0: Simple[F]) =\n new SimpleOps[F](v)\n}\n\nobject simple extends ToSimpleOps\n\ntrait SimpleSyntax[F] {\n def F: Simple[F]\n implicit def ToSimpleOps(v: F): SimpleOps[F] =\n new SimpleOps[F](v)(SimpleSyntax.this.F)\n}\n\n// New definition of our Simple typeclass that provides an instance of\n// SimpleSyntax\ntrait Simple[F] { self =\u003e\n def simplify(f: F): String\n val simpleSyntax = new SimpleSyntax[F] { def F = Simple.this }\n}\n```\n\nHere we've provided two syntax operators as aliases to\n`simplify` (because no type class is legit without unicode\noperator aliases).\n\n```scala\nimport simple._\n\n1 ⬈\n//=\u003e Simplified Int with value of 1\n\nnew C(\"I am C\") ⬈\n//=\u003e Simplified: C[name=I am C]\n\nList(1,2,3) ⬈\n//=\u003e Simplified Int with value of 1\n//=\u003e Simplified Int with value of 2\n//=\u003e Simplified Int with value of 3\n\n// boring\nnew C(\"bar\") /^\n//=\u003e Simplified: C[name=bar]\n```\n\n## Conclusion\n\nType classes are a powerful method of adding support for a set of operations on\nan existing type in an ad hoc manner. Because Scala doesn't encode type classes\nat the language level, there is a bit of boilerplate and implicit trickery to\ncreate your own type classes (compare this with Haskell, where classes are\nelegantly supported at the language level) and operator syntax.\n\n[View the full code listing for this post](https://gist.github.com/devth/735ddd34e8f29fc6b872).\n\n### Further reading\n\n- [Introduction to Typeclasses in Scala (2013)](http://tpolecat.github.io/2013/10/12/type class.html)\n- [Types and Typeclasses — Learn you a Haskell](http://learnyouahaskell.com/types-and-typeclasses)\n- [Simalacrum](https://github.com/mpilquist/simulacrum) — a modern, concise type\n class encoding for Scala using annotations\n- [Typeclassopedia](https://wiki.haskell.org/Typeclassopedia) — overview of\n Haskell's type classes, many of which are also represented in Scalaz, along\n with a very useful diagram showing relationships between type classes.\n","data":{"layout":"article","title":"A Simple type class","categories":"scala","excerpt":"Type classes provide a way of achieving ad hoc polymorphism","comments":true,"image":{"feature":"typeclass_wide.jpg","teaser":"typeclass_teaser.jpg","caption":"Schilling Cider House, Seattle, WA"}},"dateString":"Mon Aug 3 2015","slug":"a-simple-type-class","filePath":"2015-08-03-a-simple-type-class.mdx"},{"content":"\n[Yetibot](http://yetibot.com) is now [on\nDocker](https://hub.docker.com/u/yetibot)! This is the fastest way\nto get up and running. To demonstrate, let's run it with Docker using the most\nminimal configuration possible: an in-memory (non-durable) Datomic configuration\nand a single IRC adapter config. (I'm assuming your local Docker is\n[installed](https://www.docker.com/docker-toolbox).)\n\n**Update 2024-05-01:** This configuration is outdated and no longer valid.\nPlease refer to the [officiel Yetibot Ops guide](https://yetibot.com/ops-guide)\nfor the most up-to-date instructions.\n\n```bash\nmkdir -p ~/tmp/config\n\ncat \u003c\u003c EOF \u003e ~/tmp/config/config.edn\n{:yetibot\n {:db {:datomic-url \"datomic:mem://yetibot\"}\n :adapters\n [{:name \"freenode-irc\",\n :type :irc,\n :host \"chat.freenode.net\",\n :port \"6665\",\n :username \"yetibot-docker\"\n :rooms #{\"#yetibot\"}}]}}\nEOF\n\ndocker run --name yetibot \\\n -d -p 3000:3000 \\\n -v ~/tmp/config:/usr/src/app/config \\\n devth/yetibot\n\ndocker logs -f yetibot\n```\n\n**N.B.** I chose `~/tmp/config` because:\n\n\u003e If you are using Docker Machine on Mac or Windows, your Docker daemon has only\n\u003e limited access to your OS X or Windows filesystem. Docker Machine tries to\n\u003e auto-share your /Users (OS X) or C:\\Users (Windows) directory.\n\u003e — [Mount a host directory as a data volume](https://docs.docker.com/engine/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume)\n\nIf you're not using Docker Machine feel free to put it wherever you like.\n\nHop on Freenode and join the #yetibot channel. Once `yetibot-docker` joins, try\nit out!\n\n```\n!list yetibot on docker | xargs echo ⚡️⚡️⚡️ %s ⚡️⚡️⚡️\n```\n\n\n\u003cimg src=\"/images/yetibot-on-docker-irc.png\" /\u003e\n\nWhen you're done, clean up with:\n\n```bash\ndocker rm -f yetibot\n```\n\nTo get the most out of Yetibot you'll want to configure some of the optional\nservices. See the full [config\nsample](https://github.com/devth/yetibot/blob/master/config/config-sample.edn)\nand check out [yetibot.com](http://yetibot.com) for more info on the cool things\nyou can do with Yetibot!\n\n\u003csmall\u003e\u003csub\u003ewhere 𝓧 ≈ somewhere between 1 and 3.5 minutes\u003c/sub\u003e\u003c/small\u003e\n","data":{"layout":"article","title":"Yetibot on Docker in 𝓧 minutes or less","categories":"yetibot","comments":true,"excerpt":"Yetibot is now on Docker!","image":{"feature":"on_docker_wide.jpg","teaser":"on_docker_teaser.jpg","caption":"Yetibot Source Code, January 2016"}},"dateString":"Wed Jan 6 2016","slug":"yetibot-on-docker","filePath":"2016-01-06-yetibot-on-docker.mdx"},{"content":"\nWhy resort to the complexities of dynamic code generation and compilation at\nruntime?\n\n**Speed**.\n\nTo demonstrate, lets build an interpreter that runs projection and filtering on\na simulated database, then build a compiled version and look at some performance\nnumbers.\n\nThis is our goal: **compile a data structure representing a query into native\ncode to speed up a query loop**. We'll look at two specific code-generation\ntools to achieve our goal: ASM and Scala Quasiquotes.\n\nASM has been used by Java developers for years for all sorts of codegen\npurposes. It's is very fast and has low-memory requirements, but it's also very\nlow-level, making it time-consuming and tedious to use and very difficult to\ndebug.\n\nQuasiquotes are a new Scala tool for code generation. They are similar to\nmacros, except they can be used at runtime whereas macros are compile-time only.\nThey're much higher-level than ASM, so we'll compare benchmarks against the two\nto see what cost these high-levelel semantics incur, if any.\n\n## A query system\n\nFirst let's define our \"database\" consisting of a single dataset, represented as\na simple in-memory data structure using tuples as rows.\n\n```scala\n// Store a mapping of column name to ordinal for fast projection\nval schema = Seq(\"name\", \"birthYear\", \"dissertation\").zipWithIndex.toMap\n// schema: scala.collection.immutable.Map[String,Int] =\n// Map(name -\u003e 0, birthYear -\u003e 1, dissertation -\u003e 2)\n\nval db = Seq(\n (\"John McCarthy\", 1927, \"Projection Operators and Partial Differential Equations.\"),\n (\"Haskell Curry\", 1900, \"Grundlagen der kombinatorischen Logik\"),\n (\"Philip Wadler\", 1956, \"Listlessness is Better than Laziness\"),\n (\"Alonzo Church\", 1903, \"Alternatives to Zermelo's Assumption\"),\n (\"Alan Turing\", 1912, \"Systems of Logic based on Ordinals\")\n)\n```\n\nNext, we need structures that hold:\n\n1. fields to be projected\n1. filter expression tree\n\nAn efficient representation of projection fields is simply storing the ordinals.\nFor example, this is how we'd represent a projection of the name and\ndissertation columns:\n\n```scala\nval projections = Seq(0, 2)\n```\n\nThe filter expression is a little more complicated, since it's treeish in\nnature. Even though we're not supporting SQL, let's use it to help understand:\n\n```sql\nWHERE name != null AND birthYear \u003c 1910\n```\n\nA tree is a natural way to represent this syntax in abstract form:\n\n\u003cimg src=\"/images/querytree.svg\" /\u003e\n\nIn Scala, we can represent this with a Scalaz `Tree[String]`.\n\n```scala\ntype FilterExpr = Tree[String]\n\nval filterExpr: FilterExpr = And.node(\n IsNotNull.node(\n Item.node(\"name\".leaf)),\n LessThan.node(\n Item.node(\"birthYear\".leaf),\n Literal.node(\"1910\".leaf)))\n```\n\nThe root node of each tree or subtree is always an operator. The sub-forests are\nthe operands, which may be made up of operator trees.\n\nUsing this we can build a filter interpreter that evaluates whether a row should\nbe included. Some caveats about limitations of this simple example:\n\n- We'll run filtering before projection. In reality, which to run first is a\n decision that should be made by a query optimizer.\n- I'm only going to implement a few of the most common operators.\n- The filter interpreter is doing its own projection, which may overlap with the\n projected fields requested by the user and create duplicate work.\n- I'm only supporting `Int` for comparison, and I'm doing some\n nasty type casting. IRL, we'd keep better track of the types with a full blown\n schema.\n\nThese are all issues I won't address.\n\nHere's our limited set of operators:\n\n```scala\nobject Operators {\n val And = \"AND\"\n val Or = \"OR\"\n val IsNotNull = \"IS_NOT_NULL\"\n val LessThan = \"LESS_THAN\"\n val Item = \"ITEM\"\n val Literal = \"LITERAL\"\n}\nimport Operators._\n```\n\nAnd the interpreter itself:\n\n```scala\nclass FilterInterpreter(expr: FilterExpr, schema: Map[String, Int]) {\n import Operators._\n /** return true if row is filtered out by expr */\n def isFiltered(row: Product): Boolean = evalFilterOn(row)\n\n private def evalFilterOn(row: Product): Boolean = {\n def eval(expr: FilterExpr): Boolean = {\n val (operator, operands: Stream[FilterExpr]) = (expr.rootLabel, expr.subForest)\n operator match {\n case And =\u003e operands.forall(eval)\n case Or =\u003e operands.exists(eval)\n case IsNotNull =\u003e valueFor(operands(0)) != null\n case LessThan =\u003e {\n val (x :: y :: _) = operands.map(o =\u003e valueFor(o).toString.toInt).toList\n x \u003c y\n }\n }\n }\n def valueFor(node: FilterExpr) = node.rootLabel match {\n // Item expects a single operand\n case Item =\u003e row.productElement(schema(node.subForest.head.rootLabel))\n // Literal expects a single operand\n case Literal =\u003e node.subForest.head.rootLabel\n }\n eval(expr)\n }\n}\n```\n\nAnd finally, a simple query loop:\n\n```scala\ndef query(projections: Seq[Int], filterExpr: FilterExpr): Seq[Seq[Any]] = {\n val filterer = new FilterInterpreter(filterExpr, schema)\n db.flatMap { row =\u003e\n // Filter\n if (filterer.isFiltered(row)) None\n // Project\n else Some(projections.map(row.productElement))\n }\n}\n```\n\n_NB: we could use [ScalaBlitz](https://scala-blitz.github.io/) to optimize that\n`flatMap` if this was real and we were ultra-concerned about\nperformance._\n\nNotice how we return `Seq[Seq[Any]]`. At compile-time we don't know how to\ntypefully represent a row so we have to resort to the lowest-common type,\n`Any`. This is another issue that we'll fixup later in the\ncompiled version.\n\nWith all this in place, let's run it:\n\n```scala\n// SELECT name, dissertation\nval projections = Seq(0, 2)\n\n// WHERE name != null AND birthYear \u003c 1910\nval filterExpr: FilterExpr = And.node(\n IsNotNull.node(\n Item.node(\"name\".leaf)),\n LessThan.node(\n Item.node(\"birthYear\".leaf),\n Literal.node(\"1910\".leaf)))\n\nval result = query(projections, filterExpr)\nresult.map(println)\n\n//=\u003e List(John McCarthy, Projection Operators and Partial Differential Equations.)\n//=\u003e List(Philip Wadler, Listlessness is Better than Laziness)\n//=\u003e List(Alan Turing, Systems of Logic based on Ordinals)\n```\n\nSo far so good. Next up, let's dive into some bytecodes to gain a basic\nunderstanding of what's going on when we generate code on the JVM.\n\n## Bytecode primer\n\nLet's start by looking at the bytecode for a minimum viable Scala class:\n\n```scala\nclass Foo\n```\n\nCompile it with `scalac` then view the bytecode with `java -c`:\n\n```bash\nscalac Foo.scala\njavap -c Foo.class\n```\n\n```java\npublic class Foo {\n public Foo();\n Code:\n 0: aload_0\n 1: invokespecial #12 // Method java/lang/Object.\"\u003cinit\u003e\":()V\n 4: return\n}\n```\n\nFrom the [Java bytecode instructions listings\nreference](http://en.wikipedia.org/wiki/Java_bytecode_instruction_listings) we\ncan find out the meaning of these bytecode instructions:\n\n- `aload_0` — load a reference onto the stack from local variable 0 (local var 0\n is always `this`)\n- `invokespecial` — invoke instance method on object objectref (this would be\n the object that we just loaded onto the stack) and puts the result on the\n stack (might be void)\n- `return` — return void from method\n\nThis is the generated constructor for `Foo`. Since nothing\nelse is going on, it simply returns void after calling the constructor. Now\nwhat if we actually do something, like instantiate a `Foo`?\n\n```scala\nclass Foo\n\nobject RunFoo {\n val f = new Foo\n}\n```\n\nCompiling this yields a single `Foo.class` identical to the one above along with\ntwo class files: `RunFoo.class` and `RunFoo$.class`.\n\n```java\n// RunFoo.class\npublic final class RunFoo {\n public static Foo f();\n Code:\n 0: getstatic #16 // Field RunFoo$.MODULE$:LRunFoo$;\n 3: invokevirtual #18 // Method RunFoo$.f:()LFoo;\n 6: areturn\n}\n```\n\n```java\n// RunFoo$.class\npublic final class RunFoo$ {\n public static final RunFoo$ MODULE$;\n\n public static {};\n Code:\n 0: new #2 // class RunFoo$\n 3: invokespecial #12 // Method \"\u003cinit\u003e\":()V\n 6: return\n\n public Foo f();\n Code:\n 0: aload_0\n 1: getfield #17 // Field f:LFoo;\n 4: areturn\n}\n```\n\nThe JVM runs these opcodes in a stack machine: values are pushed on to the stack\nthen used as operands to later operations. For example, this is how you could\nadd two constants:\n\n```java\nbipush 28\nbipush 14\niadd\n```\n\n1. Push 28 onto the stack with `bipush`\n1. Push 14 onto the stack with `bipush`\n1. Execute `iadd` which adds two ints: it pops two values off the stack to use\n as its operands: first 14, then 28. It adds those two operands and pushes the\n result, 28, onto the stack.\n\nAt this point we could work with the new 42 value on the stack. This is how we\nwould check that the value is indeed 42:\n\n```java\n20: bipush 42\n22: if_icmpne 28\n24: ldc #3\n26: goto 32\n28: ldc #4\n32: ...\n```\n\n1. Push the value 42 onto the stack\n1. Use `if_icmpne` to compare two values from the stack. If they are not equal,\n jump to position 28, which pushes constant `#4` onto the stack using `ldc`.\n If they are equal, the next code is executed, which instead pushes constant\n `#3` onto the stack, then jumps to position 32.\n\nTedious, but simple.\n\nThis primer is only intended to wet your feet. If you want to learn more about\nbytecode, see the [Further reading](#further-reading) section at the end of this\npost.\n\n## ASM\n\nASM is a bytecode manipulation framework. It's one of several options for\nmanipulating bytecode, but I chose it because it's one of the most mature,\nrequires the least amount of memory, and it's very fast. The downside is it's\nalso quite low level. If you aren't super-concerned with performance, you\nshould check out other options, like\n[Javassist](http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/), which is much\neasier to work with.\n\nNow, to use ASM, the more familiar you are with bytecode the better off you'll\nbe, but for newbs like us, there is ASMifier, which takes a compiled class and\ngenerates the ASM code for it. I'm going to avoid Scala for this excercise,\nsince Java maps more closely to bytecode, and we can use the resulting bytecode\nfrom Scala either way. I want to use Tuples to represent fully typed rows, so\nlet's see what the ASM code looks like for this Java class:\n\n```java\nimport scala.Tuple2;\n\npublic class TupleFromJava {\n\n Tuple2\u003cString, Integer\u003e tup = new Tuple2\u003cString, Integer\u003e(\"foo\", 2);\n\n public TupleFromJava() {\n }\n\n public Tuple2\u003cString, Integer\u003e getTup() {\n return tup;\n }\n\n}\n```\n\nCompile it, making sure the scala-lib jar is on your classpath:\n\n```bash\njavac -cp $SCALA_LIB TupleFromJava.java\n```\n\nThen use the ASMifier on the classfile:\n\n```bash\njava -cp asm-5.0.3/lib/all/asm-all-5.0.3.jar \\\n org.objectweb.asm.util.ASMifier TupleFromJava.class\n```\n\nHere is the generated ASM code, heavily annotated with comments. It's a little\ntedious to work through, but if you really want to understand bytecode and ASM,\nI encourage you to read the comments and work through every line until it's\ninternalized and you feel comfortable with it.\n\n```java\nimport java.util.*;\nimport org.objectweb.asm.*;\npublic class TupleFromJavaDump implements Opcodes {\n\n public static byte[] dump () throws Exception {\n\n ClassWriter cw = new ClassWriter(0);\n FieldVisitor fv;\n MethodVisitor mv;\n AnnotationVisitor av0;\n\n // Generate a public class inheriting from java.lang.Object\n cw.visit(52, ACC_PUBLIC + ACC_SUPER, \"TupleFromJava\", null,\n \"java/lang/Object\", null);\n\n // Initialize the `tup` field with a null value. When fields are declared in\n // Java classes, they aren't fully initialized until the constructor runs.\n // Note the format of the string representation of `Tuple2`'s constructor.\n {\n fv = cw.visitField(0, \"tup\", \"Lscala/Tuple2;\",\n \"Lscala/Tuple2\u003cLjava/lang/String;Ljava/lang/Integer;\u003e;\", null);\n fv.visitEnd();\n }\n\n // Generate the public constructor named \u003cinit\u003e by convention\n {\n mv = cw.visitMethod(ACC_PUBLIC, \"\u003cinit\u003e\", \"()V\", null, null);\n mv.visitCode();\n\n // Put `this` on the stack\n mv.visitVarInsn(ALOAD, 0);\n\n // Invoke constructor on `this`. Note that it takes no params and returns\n // void, and it consumes the `this` that we put on the stack above.\n mv.visitMethodInsn(INVOKESPECIAL, \"java/lang/Object\", \"\u003cinit\u003e\", \"()V\", false);\n\n // Put `this` on the stack again\n mv.visitVarInsn(ALOAD, 0);\n\n //\n // Start `tup` initialization {\n //\n\n // Put a new scala.Tuple2 on the stack then duplicate the object reference\n // on top of the stack. Note that since it is an object reference and not\n // the object itself, any mutation we do to the object will be reflected in\n // any references to that object on the stack.\n mv.visitTypeInsn(NEW, \"scala/Tuple2\");\n mv.visitInsn(DUP);\n\n // Push constant \"foo\" from the constant pool onto the stack\n mv.visitLdcInsn(\"foo\");\n\n // Load the int constant 2 onto the stack\n mv.visitInsn(ICONST_2);\n\n // Autobox the int in a java.lang.Integer. This pops the int off the stack\n // and replaces it with the Integer.\n mv.visitMethodInsn(INVOKESTATIC, \"java/lang/Integer\", \"valueOf\",\n \"(I)Ljava/lang/Integer;\", false);\n\n // At this point our stack looks like:\n // Integer.valueOf(2)\n // \"foo\"\n // Tuple2 (reference)\n // Tuple2 (reference)\n\n // Initialize the Tuple2. Looking at the signature, we can see it takes\n // two java.lang.Objects (instead of a String and Integer, due to type\n // erasure), which it will consume from the stack in addition\n // to one of the Tuple2 references.\n mv.visitMethodInsn(INVOKESPECIAL, \"scala/Tuple2\", \"\u003cinit\u003e\",\n \"(Ljava/lang/Object;Ljava/lang/Object;)V\", false);\n\n // Finally, we store our Tuple2 value in the `tup` field. This consumes\n // the second copy of the Tuple2 reference we had on the stack.\n mv.visitFieldInsn(PUTFIELD, \"TupleFromJava\", \"tup\", \"Lscala/Tuple2;\");\n\n //\n // End `tup` initialization }\n //\n\n // Constructors always return void because constructors should only\n // initialize the object and do nothing else.\n mv.visitInsn(RETURN);\n\n // Sets the max stack size and max number of local vars. This can be\n // calculated automatically for you if you use COMPUTE_FRAMES or COMPUTE_MAXS\n // in the ClassWriter constructor.\n mv.visitMaxs(5, 1);\n mv.visitEnd();\n }\n\n // Create a public method `getTup` which takes no args and returns a\n // scala.Tuple2\n {\n mv = cw.visitMethod(ACC_PUBLIC, \"getTup\", \"()Lscala/Tuple2;\",\n \"()Lscala/Tuple2\u003cLjava/lang/String;Ljava/lang/Integer;\u003e;\", null);\n mv.visitCode();\n // Load `this` onto the stack\n mv.visitVarInsn(ALOAD, 0);\n // Load a reference to the `tup` field on `this` onto the stack, consuming\n // `this` in the process\n mv.visitFieldInsn(GETFIELD, \"TupleFromJava\", \"tup\", \"Lscala/Tuple2;\");\n // Return the reference to the `tup` field\n mv.visitInsn(ARETURN);\n // We only needed a max stack size of 1 and maximum of 1 local vars\n mv.visitMaxs(1, 1);\n mv.visitEnd();\n }\n\n // Finish writing the class\n cw.visitEnd();\n\n return cw.toByteArray();\n\n }\n}\n```\n\nNot too bad. ASMifier helpfully wraps each logical chunk in blocks. The first\nblock creates the `tup` field. The second block is our public\nconstructor. It calls super, which invokes the constructor on the\n`java.lang.Object` superclass, then initializes the\n`tup` field, and finally returns void. The third block\nis the `getTup` getter.\n\n_NB: If you're having trouble following this, I recommend generating some ASM\ncode with ASMifier, then annotating it yourself. It really helps to internalize\nthe JVM bytecodes and how to work on a stack machine._\n\nInside a method, the parameters can be referenced by corresponding local\nvariable indices. For example:\n\n```scala\ndef add(x: Int, y: Int)\n```\n\nIn this case, `x` is available at `1` and\n`y` is available at `2`. To load and use\nthese, we would use `visitVarInsn` which visits a local\nvariable instruction. Using ASM, this is how we'd add `x` and\n`y` and store the result in local variable\n`3`:\n\n```java\nmv.visitVarInsn(ILOAD, 1); // Load x onto the stack\nmv.visitVarInsn(ILOAD, 2); // Load y onto the stack\nmv.visitInsn(IADD); // Pop two values off the stack, add them,\n // then put the result back on the stack\nmv.visitVarInsn(ISTORE, 3); // Pop a value off the stack and store\n // it in local variable 3\n```\n\nWhen you generate ASM using ASMifier it can generate all the labels and local\nvar mappings, which is necessary information for debuggers to show you the\ncorrect names of the local variables in a given stack, since in the JVM indices\nare used instead of names. When writing by hand, you could opt to not write\nthese instructions, or add them later if you need them.\n\n## Compiled queries with ASM\n\nLet's use this knowledge to compile the query we originally interpreted. To\nstart, let's write a fast but static version of the query so we can quickly\nfigure out which parts need to be made dynamic. When using ASM it's a good idea\nto distil the essense of which part needs to be made dynamic so the generator\ncode ends up being as small as possible. Since ASM is hard to read, write, and\ndebug, this is very important.\n\nLet's perform the same projection and filtering we used before, but this time\nwithout the interpretation.\n\n```scala\nobject StaticQuery {\n\n def query(db: Seq[(String, Int, String)]) = {\n db.flatMap { case (name, birthYear, dissertation) =\u003e\n if (birthYear \u003c 1910 \u0026\u0026 name != null) Some((name, dissertation))\n else None\n }\n }\n\n}\n```\n\nThis should be _much_ faster than our interpreted query because it's running a\nsimple conditional instead of walking and evaluating a\n`FilterExpr` on every row.\n\nLet's construct a quick benchmark using\n[ScalaMeter](http://scalameter.github.io/) to verify our assumptions.\n\n```scala\nobject QueryBenchmark extends PerformanceTest.Quickbenchmark {\n\n val birthYears = Gen.range(\"birthYears\")(9990, 10000, 1)\n\n val records = for {\n birthYear \u003c- birthYears\n } yield (0 until birthYear).map((\"name\", _, \"diss\"))\n\n performance of \"Querying\" in {\n measure method \"StaticQuery\" in {\n using (records) in { StaticQuery.query(_) }\n }\n\n measure method \"InterpretedQuery\" in {\n using (records) in {\n InterpretedQuery.query(_, Main.projections, Main.filterExpr)\n }\n }\n }\n\n}\n```\n\nResults from the last result of each measurement:\n\n- `StaticQuery`: 1.329709ms\n- `InterpretedQuery`: 6.949921ms\n\nThe static query is between 5 and 6 times faster than interpreting.\n\nLet's rewrite `StaticQuery` in Java and use it as a template\nfor compiling queries. There are at least two reasons why I significantly\ndislike running ASMifier against Scala classes:\n\n1. It generates a gnarly blob of unreadable bytes for the ScalaSignature (which\n is apparently used to store Scala-specific bits in class files and is\n required for reflection and for compiling against).\n1. Scala objects and methods get split into separate class files when they're\n compiled, making it hard to stitch together the results with multiple\n ASMifier runs.\n\n```java\nimport scala.Tuple3;\nimport scala.collection.Seq;\nimport scala.collection.mutable.ArrayBuffer;\nimport scala.collection.Iterator;\n\npublic class StaticJavaQuery {\n\n public static Seq\u003cTuple3\u003cString, Integer, String\u003e\u003e query(\n Seq\u003cTuple3\u003cString, Integer, String\u003e\u003e db) {\n Iterator\u003cTuple3\u003cString, Integer, String\u003e\u003e iter = db.iterator();\n ArrayBuffer\u003cTuple2\u003cString, String\u003e\u003e acc =\n new ArrayBuffer\u003cTuple2\u003cString, String\u003e\u003e();\n while (iter.hasNext()) {\n Tuple3\u003cString, Integer, String\u003e row = iter.next();\n Integer birthYear = row._2();\n if (birthYear.intValue() \u003c 1910 \u0026\u0026 row._1() != null) {\n acc.$plus$eq(new Tuple2\u003cString, String\u003e(\n row._1(),\n row._3()\n ));\n }\n }\n return acc;\n }\n\n}\n```\n\nIt's not pretty, but it works, and it happens to be even faster than the static\nScala query. We'll start by feeding it to ASMifier, convert the output to Scala,\nthen work on making the result dynamic. Converted output, heavily annotated:\n\n```scala\nimport org.objectweb.asm._, Opcodes._\nimport Database.FilterExpr\n\nobject CompiledQueryGen extends Opcodes {\n\n def generate: Array[Byte] = {\n\n val cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES)\n var mv: MethodVisitor = null\n\n cw.visit(52, ACC_PUBLIC + ACC_SUPER, \"CompiledQuery\", null,\n \"java/lang/Object\", null)\n\n // Constructor\n {\n mv = cw.visitMethod(ACC_PUBLIC, \"\u003cinit\u003e\", \"()V\", null, null)\n mv.visitCode()\n mv.visitVarInsn(ALOAD, 0)\n mv.visitMethodInsn(INVOKESPECIAL, \"java/lang/Object\", \"\u003cinit\u003e\", \"()V\", false)\n mv.visitInsn(RETURN)\n mv.visitMaxs(1, 1)\n mv.visitEnd()\n }\n\n // Static query method\n {\n mv = cw.visitMethod(ACC_PUBLIC + ACC_STATIC, \"query\", \"(Lscala/collection/Seq)Lscala/collection/Seq\", \"(Lscala/collection/Seq\u003cLscala/Tuple3\u003cLjava/lang/StringLjava/lang/IntegerLjava/lang/String\u003e\u003e)Lscala/collection/Seq\u003cLscala/Tuple3\u003cLjava/lang/StringLjava/lang/IntegerLjava/lang/String\u003e\u003e\", null)\n mv.visitCode()\n\n // Load the `db` argument onto the stack\n mv.visitVarInsn(ALOAD, 0)\n // Invoke the `iterator` method on db, putting the iterator on the stack\n mv.visitMethodInsn(INVOKEINTERFACE, \"scala/collection/Seq\", \"iterator\", \"()Lscala/collection/Iterator\", true)\n // Store the iterator object at index 1\n mv.visitVarInsn(ASTORE, 1)\n\n // Stack size = 0\n\n // Instantiate a new ArrayBuffer `acc` {\n mv.visitTypeInsn(NEW, \"scala/collection/mutable/ArrayBuffer\")\n // Duplicate the reference to it on the stack\n mv.visitInsn(DUP)\n // Initialize the `acc` ArrayBuffer\n mv.visitMethodInsn(INVOKESPECIAL, \"scala/collection/mutable/ArrayBuffer\", \"\u003cinit\u003e\", \"()V\", false)\n // Store the ArrayBuffer at index 2\n mv.visitVarInsn(ASTORE, 2)\n\n // Stack size = 0\n\n // A label is a point we can jump to with GOTO-style instructions. l0\n // marks the start of the while loop. The point at which the label is\n // visited represents its position and l0 is visited immediately.\n // while (...) {\n val l0 = new Label\n mv.visitLabel(l0)\n\n // Check the while condition\n // Load the iterator onto the stack from index 1\n mv.visitVarInsn(ALOAD, 1)\n // Call `hasNext` on the iterator, storing the boolean result on the\n // stack. The JVM stores boolean as int: 0 is false, 1 is true.\n mv.visitMethodInsn(INVOKEINTERFACE, \"scala/collection/Iterator\",\n \"hasNext\", \"()Z\", true)\n\n // Stack size = 1, hasNext boolean\n\n // Create another jump location for the end of the loop. l1 isn't visited\n // until later at the end of the loop body but we need to create the label\n // here in order to reference it in `IFEQ`.\n val l1 = new Label\n // A jump instruction with IFEQ (\"if equals\") checks the current value on\n // the stack. If it's 0 (false) it jumps to the label, thus ending our\n // while loop.\n mv.visitJumpInsn(IFEQ, l1)\n\n // Stack size = 0\n\n // Load iterator onto the stack again\n mv.visitVarInsn(ALOAD, 1)\n // Obtain the `row` value from the iterator\n mv.visitMethodInsn(INVOKEINTERFACE, \"scala/collection/Iterator\", \"next\", \"()Ljava/lang/Object\", true)\n // Ensure the value is of expected type, Tuple3. This instruction pops a\n // value off the stack, checks it, then puts it back on the stack.\n mv.visitTypeInsn(CHECKCAST, \"scala/Tuple3\")\n // Store the row Tuple3 at local variable index 3\n mv.visitVarInsn(ASTORE, 3)\n // Load it again\n mv.visitVarInsn(ALOAD, 3)\n\n // Stack size = 1, row Tuple3\n\n // Invoke the `_2` method on the row to get the birthYear\n mv.visitMethodInsn(INVOKEVIRTUAL, \"scala/Tuple3\", \"_2\", \"()Ljava/lang/Object\", false)\n // Ensure the expected type, Integer\n mv.visitTypeInsn(CHECKCAST, \"java/lang/Integer\")\n // Store birthYear at local var 4\n mv.visitVarInsn(ASTORE, 4)\n // Load birthYear from local var 4\n mv.visitVarInsn(ALOAD, 4)\n // Invoke the `intValue` method on birthYear\n mv.visitMethodInsn(INVOKEVIRTUAL, \"java/lang/Integer\", \"intValue\", \"()I\", false)\n // Push a short constant on to the stack\n mv.visitIntInsn(SIPUSH, 1910)\n\n // Stack size = 2, birthYear int, 1910 short\n\n // Any time we need to branch in some way, we need labels and jump\n // instructions. l2 marks the end of the filtering if statement, allowing\n //\n // us to jump over the body.\n val l2 = new Label()\n\n // Jump instructions are always the inverse predicate because if it\n // evaluates to true then it jumps, skipping the body of the if block.\n // IF_ICMPGE is short for \"if int compare greater than or equal\", so:\n // If value1 \u003e= value2 then jump to l2, where\n // value1 = birthYear\n // value2 = 1910\n mv.visitJumpInsn(IF_ICMPGE, l2)\n // That's the first half of the if predicate. Now we check the other half.\n\n // Load the row Tuple3\n mv.visitVarInsn(ALOAD, 3)\n // Invoke the `_1` method on the row to get the name String\n mv.visitMethodInsn(INVOKEVIRTUAL, \"scala/Tuple3\", \"_1\", \"()Ljava/lang/Object\", false)\n // This condition is much simpler and the JVM even has an instruction to\n // check for null. If the name String is null, jump to l2.\n mv.visitJumpInsn(IFNULL, l2)\n\n // Body of the if block {\n\n // Load the `acc` ArrayBuffer\n mv.visitVarInsn(ALOAD, 2)\n // Load the `row` Tuple3\n mv.visitVarInsn(ALOAD, 3)\n // Invoke the `$plus$eq` method on `acc` which mutates it, appending the\n // `row` Tuple3, and stores the result (which is simply itself) on the\n // stack.\n mv.visitMethodInsn(INVOKEVIRTUAL, \"scala/collection/mutable/ArrayBuffer\", \"$plus$eq\", \"(Ljava/lang/Object)Lscala/collection/mutable/ArrayBuffer\", false)\n // Discard the last item on the stack since we no longer need it.\n mv.visitInsn(POP)\n\n // }\n\n // Mark the end of the if block\n mv.visitLabel(l2)\n\n // Jump back to the start of the while loop\n mv.visitJumpInsn(GOTO, l0)\n // Mark the end of the while loop\n mv.visitLabel(l1)\n // } // end while\n\n // Load the acc Tuple3\n mv.visitVarInsn(ALOAD, 2)\n // Return the object on the stack\n mv.visitInsn(ARETURN)\n // Compute the max stack size and number of local vars (computed\n // automatically for us via COMPUTE_FRAMES)\n mv.visitMaxs(0, 0)\n // End the method\n mv.visitEnd()\n }\n\n // End the class\n cw.visitEnd()\n\n // Return the bytes representing a generated classfile\n cw.toByteArray\n }\n}\n```\n\nSince we're using `ClassWriter.COMPUTE_FRAMES` I was able to\nremove all the `visitFrame` calls that ASMifier generated. I\nalso deleted the generated `FieldVisitor` and\n`AnnotationVisitor` as they were both unused. I used `0` for\nall arguments to `visitMaxs` as\n`COMPUTE_FRAMES` implies `COMPUTE_MAXS`, which still requires\ncalls to `visitMaxs` but ignores the arguments.\n\n## Scala Quasiquotes\n\nThis part of the post is not yet written. The intent was to explore Scala\nQuasiquotes facilities for codegen.\n\n- [Quasiquotes](http://docs.scala-lang.org/overviews/quasiquotes/intro.html).\n- [Generative Programming Basics](https://scala-lms.github.io/tutorials/02_basics.html#__toc_id:54231)\n- [Introduction to code generation with scalameta](http://www.michaelpollmeier.com/2016/12/01/scalameta-code-generation-tutorial)\n\n## Next steps\n\nAn interesting direction to take this, now that we have a foundation for\ndynamically compiling queries, would be to add a SQL interface. [Apache\nCalcite](https://calcite.apache.org/) is well-suited to do just that.\n\n## Further reading\n\n- [Java bytecode instruction listings (Wikipedia - useful reference)](http://en.wikipedia.org/wiki/Java_bytecode_instruction_listings)\n- [Secrets of the Bytecode Ninjas (InfoQ)](http://www.infoq.com/articles/Secrets-of-the-Bytecode-Ninjas)\n- [Hacking Java Bytecode for Programmers (part 1 of a 4 part series)](http://www.acloudtree.com/hacking-java-bytecode-for-programmers-part1-the-birds-and-the-bees-of-hex-editing/)\n- [java-bytecode-asm tag on StackOverflow](http://stackoverflow.com/questions/tagged/java-bytecode-asm)\n- [JVM Internals (blog post)](http://blog.jamesdbloom.com/JVMInternals.html)\n- [3 approaches to Scala code generation](http://yefremov.net/blog/scala-code-generation/)\n","data":{"layout":"article","title":"Compiled queries in Scala","categories":"scala","comments":true,"toc":true,"image":{"feature":"compiled-queries-feature.jpg","teaser":"compiled-queries-teaser.jpg","caption":"Northwest San Francisco and the Golden Gate strait from the Hamon Observation Tower at de Young Museum. February 2017"},"excerpt":"Compile a data structure representing a query into native\ncode to speed up a query loop using ASM.\n"},"dateString":"Sun Feb 26 2017","slug":"compiled-queries-in-scala","filePath":"2017-02-26-compiled-queries-in-scala.mdx"},{"content":"\nThere are many [too-strong\nopinions](https://hn.algolia.com/?query=environment%20variables\u0026sort=byDate\u0026prefix\u0026page=0\u0026dateRange=all\u0026type=story)\non the \"right\" way to do configuration but they primarily come down to these\noptions:\n\n1. flat files in some known format like JSON, YAML, properties files, EDN, etc.\n1. environment variables\n1. CLI arguments at start up\n1. config from a data store (which ironically requires separate initial config\n in one of the above methods in order to access the data store)\n\nWhile #4 is an interesting option in light of modern cluster-based infrastructure\nand tools like `consul` and `etcd`, we're going to focus on the first three\nmechanisms for this post: flat files, env vars and CLI arguments.\n\n**Systems should not require configuration via only one mechanism, but instead\nallow any or all available mechanisms in combination.** This affords the\noperator of the system to configure it in the way that's most suitable according\nto their infrastructure, preferences, policies, and politics.\n\nWhen using multiple mechanisms there needs to be a very clear priority in which one\nmechanism overrides the other. Typically env vars override flat file config, as\nthey are more ephemeral and easy to set.\n\nA few modern examples of systems that work this way today include:\n\n- [ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html)\n- HashiCorp tools like [Consul](https://www.consul.io/docs/agent/options.html)\n- TICK stack e.g. [Chronograf](https://docs.influxdata.com/chronograf/v1.6/administration/config-options#chronograf-service-options)\n\n## Symmetry\n\nA problem that immediately comes to light once we say that any individual value\ncan be configured by any available mechanism is the representational difference\nbetween mechanisms:\n\n- Env vars and CLI args are stringly typed key value pairs\n- Config files support arbitrarily-deep tree structures with varying support for\n primitive and collection types like strings, numbers, booleans, arrays and\n maps\n\nBecause of these intrinsic significant structural differences we need a\ntranslation layer between KV pair and tree. If we adopt a single constraint the\ntranslation layer becomes simpler to express: **All leafs in the tree are\nstrings.**\n\nThe implication is that our system which consumes the config must parse the\nstring to obtain its expected format.\n\n## Introducing dec: Deep Environmental Config\n\n[dec](https://github.com/devth/dec) is a tiny library that embraces this\nconstraint and provides an `explode` function to transform KV pairs into an\nexpected shape equivalent to a potentially deep EDN structure.\n\n\u003cimg\n style={{ width: \"100%\" }}\n src=\"https://github.com/devth/dec/raw/master/img/dec-colors.png?raw=true\"\n alt=\"dec logo\"\n/\u003e\n\nThe following configurations are equivalent from the perspective of dec:\n\n```bash\nMY_DB_PORT=4567\nMY_DB_HOST=database\nMY_URL=https://my-system/\n```\n\n```clojure\n{:my\n {:db {:port \"4567\" :host \"database}\n :url \"https://my-system\"}}\n```\n\nNotice how trees are serialized into KV pairs with a simple `_` delimiter. (The\ndelimiter is customizable in `dec` but `_` is the default.)\n\nSimilarly, arrays can be represented:\n\n```bash\nMY_SERVER_0_HOST=serverA\nMY_SERVER_1_HOST=serverB\n```\n\n```clojure\n{:my\n {:server [{:host \"serverA\"} {:host \"serverB\"}]}}\n```\n\n## Schema\n\nDue to our constraint that all values be strings, it becomes important that\nthe system consuming the config strings have a valid way to parse and validate\nthe strings into actual expected type.\n\nBuilding a configuration schema into config consumers has some other nice\nproperties, particularly when using clojure.spec:\n\n- Validate expected shape of config at runtime with precise error messaging\n describing the exact location of invalid config and expected values\n- Generate config example structures from schema in any format (e.g. the full\n list of supported env vars or EDN tree)\n\n## In the wild\n\nTo demonstrate we can look at [Yetibot](https://yetibot.com), an open source\nchat bot written in Clojure that embraces the above concepts.\n\nYetibot can be configured via env, edn, or a combination of both. For example, a\nminimal Yetibot config might be:\n\n```bash\nYB_ADAPTERS_MYSLACK_TYPE=Slack\nYB_ADAPTERS_MYSLACK_TOKEN=xoxb-my-token\nYB_DB_URL=postgresql://yetibot:yetibot@postgres:5432/yetibot\n```\n\nYetibot expects all env vars to be either prefixed by `YB` or `YETIBOT`, thus\nthe following is equivalent:\n\n```bash\nYETIBOT_ADAPTERS_MYSLACK_TYPE=Slack\nYETIBOT_ADAPTERS_MYSLACK_TOKEN=xoxb-my-token\nYETIBOT_DB_URL=postgresql://yetibot:yetibot@postgres:5432/yetibot\n```\n\nAnd since it uses `dec`, an edn file is also equivalent:\n\n```clojure\n{:yetibot\n {:adapters {:myslack {:type \"Slack\" :token \"xoxb-my-token\"}}\n :db {:url \"postgresql://yetibot:yetibot@postgres:5432/yetibot\"}}}\n```\n\nNote that `edn` isn't a requirement; anything that can be parsed into Clojure\ncollections would be equivalent.\n\n## Future\n\nIt might be useful to create a library that actually slurps the config and munge\nmultiple mechanisms, using `dec`. Prefix handling and merging is slightly\ntricky. This is all currently part of\n[yetibot.core](https://github.com/yetibot/yetibot.core/blob/4b607726bae926de31a48bb8a05e7345a8668484/src/yetibot/core/config.clj#L19-L47).\nThis library should also utilize `clojure.spec` to validate the expected shape\nof a config, provide validation, friendly error messages, and config generation.\n\n## Conclusion\n\nThere are known pros and cons to both approaches. A system should not perscribe\nwhich sets of constraints to adopt, but instead allow consumers to weigh their\nown tradeoffs and allow them to run things however they want.\n\nBy adopting a simple constraint and providing a very small library we can\nsupport this.\n","data":{"layout":"article","title":"dec: Deep Environmental Config","categories":"config","comments":true,"published":true,"toc":true,"image":{"feature":"dec_wide.png","teaser":"dec_410_228.png"},"excerpt":"dec is a tiny library that embraces constraints to afford users equivalency\nbetween multiple ways of setting environment variables according to their\nparticular requirements.\n"},"dateString":"Tue Jul 10 2018","slug":"dec-deep-environmental-config","filePath":"2018-07-10-dec-deep-environmental-config.mdx"},{"content":"\nThe aim of this guide is to demonstrate the fastest way to create a client-side\nweb app and get it deployed. I'm writing it for myself in preparation for an\nupcoming [Startup Weekend\nevent](http://communities.techstars.com/usa/colorado-springs/startup-weekend/13311).\nWe'll keep it CLI-heavy and automation-friendly as opposed to point-and-clicking\nour way through web dashboards.\n\n## Tech\n\n- [GitLab](https://gitlab.com) - GitLab's unlimited private repos, amazing CI\n and free public pages make it an excellent choice for deploying static sites.\n- React - [`create-react-app`](https://github.com/facebook/create-react-app)\n is the quickest way to start hacking on React without messing around with\n Babel and Webpack configs.\n- ES6 - we'll use plain ES6 instead of something fancier like TypeScript,\n ReasonML, or Elm because the focus is speed and simplicity.\n\n## Caveats\n\n- We'll eschew typical production concerns like performance, security,\n configuration, and tests since we're aiming to build an MVP in a weekend.\n- Even though a web app will likely need some CRUD we won't spend the time and\n effort to use an actual database; instead the prototype will use client-side\n state to simulate how it will work.\n\n## Prerequisites\n\nThis guide assumes [Homebrew](https://brew.sh/) but the following packages can\nbe installed by many other means.\n\n```bash\nbrew install yarn\nbrew install jq\nbrew install ruby\n```\n\n## GitLab\n\nLet's create a GitLab group so we can collaborate with our teammates.\n\n```bash\ngem install gitlab\nexport GITLAB_API_ENDPOINT=https://gitlab.com/api/v4\n\n# visit https://gitlab.com/profile/personal_access_tokens and generate a token\necho \"Enter your GitLab token\"\n read -s gitlab_token\nexport GITLAB_API_PRIVATE_TOKEN=$gitlab_token\n\ngitlab user # make sure we're authenticated\nuser_id=`gitlab user --json | jq '.result.id'`\necho $gitlab_user_id # this should be an integer\n\ngitlab help create_group\n# pick your own unique group name and group path\n# these are global on GitLab\ngroup_json=`gitlab create_group startup-weekend-2019 startup-weekend-2019 --json`\necho $group_json | jq\ngroup_id=`echo $group_json | jq '.result.id'`\necho $group_id # should be an integer\n\n# add your teammates (you'll need their GitLab user IDs)\ngitlab help add_group_member\n# we'll give everyone owner permissions for this exercise\n# see https://docs.gitlab.com/ee/api/access_requests.html for docs\naccess_level=50\ngitlab add_group_member $group_id $user_id $access_level\n# the above will fail because we're already a member\n# instead perform this for gitlab users on your team that you want to add\n# you can do this via the web dashboard if you like\n```\n\nNow that our group is setup and members added, let's create the repo under it.\n\n```bash\ngitlab help create_project\nproject_json=`gitlab create_project app \"{namespace_id: '$group_id'}\" --json`\n# confirm it succeeded:\necho $project_json\n```\n\n## Node.js React App\n\nLet's generate a fresh React app:\n\n```bash\nyarn global add create-react-app\ncreate-react-app startup-weekend\ncd startup-weekend\nyarn install\n```\n\nNow configure the git repo we setup in the previous step and push to it:\n\n```bash\ngit remote add origin `echo $project_json | jq -r '.result.ssh_url_to_repo'`\ngit push -u origin master\n```\n\nCongratulations, your app source is now safe in GitLab. The next step is to\ndeploy!\n\n## Deployment on GitLab Pages\n\nFirst we need to specify the `homepage` property in `package.json` so CRA\ncomputes the correct asset paths when it builds assets.\n\n```bash\ngroup_path=`echo $group_json | jq -r '.result.full_path'`\nproject_path=`echo $project_json | jq -r '.result.path'`\ngitlab_pages_url=`echo https://$group_path.gitlab.io/$project_path`\n\ntmp=$(mktemp)\ncat package.json \\\n | jq --arg homepage $gitlab_pages_url '. + {homepage: $homepage}' \u003e $tmp \\\n \u0026\u0026 mv $tmp package.json\n\ngit commit -am 'Add homepage to package.json'\n```\n\nNow we can setup CI. Run this to add a `.gitlab-ci.yml` to your repo specifying\nhow to build in GitLab CI and deploy to GitLab Pages:\n\n```bash\ncat \u003c\u003c 'EOF' \u003e .gitlab-ci.yml\ncache:\n untracked: true\n key: \"$CI_BUILD_REF_NAME\"\n paths:\n - node_modules/\n - .yarn-cache\n\nimage: node\n\nstages:\n - deps\n - test\n - publish\n\ndeps:\n stage: deps\n script:\n - yarn config set cache-folder .yarn-cache\n - yarn install --pure-lockfile\n\npages: # must be named pages to publish to GitLab pages!\n stage: publish\n script:\n - yarn build\n # replace public contents with build - GitLab requires a `public` dir\n - rm -rf public/*\n - mv build/* public\n - ls -al public/\n artifacts:\n paths:\n - public\nEOF\n```\n\nWith this in place, GitLab CI will build and deploy our app on every `git push`.\nLet's try it out:\n\n```bash\ngit add .gitlab-ci.yml \u0026\u0026 git commit -m 'Setup GitLab CI deployments'\ngit push\n```\n\nOpen up the pipelines dashboard and sit back while your app is built and\ndeployed:\n\n```bash\nopen `echo $project_json | jq -r '.result.web_url'`/pipelines\n```\n\nOnce it's finished, view your app:\n\n```bash\nopen $gitlab_pages_url\n```\n\nAt this point you can start developing your app and have it auto deployed any\ntime someone pushes to `master`.\n\n## Closing thoughts\n\nOne of the [Startup Weekend\nresources](https://startupweekend.org/attendees/resources) is $3k worth of\ncredits on Google Cloud Platform. While GCP is an excellent option for real\napps, I opted to keep things as minimal as possible, and it ended up being free.\n\nWe could very easily extend this further using the [Firebase Realtime\nDatabase](https://firebase.google.com/docs/database/) and [Firebase\nAuthentication](https://firebase.google.com/docs/auth/) which gets you Google /\nFacebook / Twitter / GitHub login and a schemaless data store with very little\nwork. These options change nothing from a deployment standpoint as Firebase\nusage remains purely client side. And since Firebase is a part of the GCP\noffering you can use those credits.\n","data":{"layout":"article","title":"Fast app generation and deployment","categories":"infra","comments":true,"excerpt":"The aim of this guide is to demonstrate the fastest way to create a client-side web app and get it deployed.","published":true,"toc":true,"image":{"feature":"fast_app_feature.jpg","teaser":"fast_app_teaser.jpg","caption":"Leaving Montana"}},"dateString":"Sun Feb 3 2019","slug":"fast-app-deploy-guide-for-startup-weekend","filePath":"2019-02-03-fast-app-deploy-guide-for-startup-weekend.mdx"},{"content":"\nThese are my notes from the mind-expanding talk [\"Unison: a new distributed\nprogramming language\" by Paul\nChiusano](https://www.youtube.com/watch?v=gCWtkvDQ2ZI\u0026feature=youtu.be).\n\n\u003e Unison is an open source functional programming language with special support\n\u003e for building distributed, elastic systems. It began as an experiment: rethink\n\u003e all aspects of the programming experience, including the core language,\n\u003e runtime, tooling, as well as code versioning and publishing, and then do\n\u003e whatever is necessary to eliminate needless complexity and make building\n\u003e software once again delightful, or at the very least, reasonable.\n\u003e\n\u003e We're used to thinking of a program as a thing that describes what a single OS\n\u003e process will do, and then using a separate layer of technologies outside of\n\u003e our programming languages to \"configure\" many separate programs into a single\n\u003e distributed, elastic \"system\". This gets complicated. The core language of\n\u003e Unison starts with the premise that no matter how many nodes a computation\n\u003e occupies, it should be expressible via a single program, not many separate\n\u003e programs. Unison programs can describe their own deployment, elastically scale\n\u003e and orchestrate themselves, and deploy themselves in parallel onto any number\n\u003e of nodes for execution.\n\u003e\n\u003e This talk introduces the Unison language and its tooling and shows what it can\n\u003e be like to program systems of any size with this model of computing.\n\u003e\n\u003e Paul Chiusano\u003cbr /\u003e\n\u003e Unison Computing\u003cbr /\u003e\n\u003e @pchiusano\n\u003e\n\u003e Paul Chiusano started the research that led to the Unison language and is a\n\u003e cofounder of Unison Computing, a public benefit corp. He has over a decade of\n\u003e experience with purely functional programming in Haskell and Scala and\n\u003e coauthored the book Functional Programming in Scala. He lives and works in\n\u003e Somerville, MA.\n\nUnison is language inspired by Haskell, Erlang and Frank, designed in a way that\nlends itself to some amazing characteristics. The issues that Unison addresses\ncause tons of headache and lost hours in other languages and build systems.\n\nUnison's goal is to drastically improve the developer experience around writing,\ntesting and deploying code \"by rethinking anything and everything about\nprogramming\".\n\nThe language is based on a core technical principle: **content-addressed code**.\nDefinitions are identified by the hash of their content. Names are simply\nmetadata for the purpose of human consumption but have no affect on code that\ndepends on that function since they simply reference it by hash.\n\nSo what happens when you embrace this principle? You gain a ton of benefits,\nincluding:\n\n- **Function renames never break the rest of the code**, including downstream\n dependencies because all code is content-addressable via the hash of its\n definition. Names are just metadata attached to the hash.\n- **No builds**. Once someone adds a function to a codebase, its AST and\n metadata (like name) is built and stored in the code base, so when you\n checkout a code base everything is already built.\n- **Cached test results**. Test results can be cached because unit tests are\n pure (aka referentially transparent) and only run again if a function being\n called by the test changes.\n- **Codebase is append only**. Definitions are hashed, so if a definition\n changes a new hash is created and appended to the codebase. The cache never\n needs to be invalidated.\n- **Code is indexed in interesting ways**, such as by types like `Nat -\u003e [a] -\u003e\n[a]` which yields `List.drop` and `List.take`.\n- **Solves the diamond dependency problem** (e.g. `D` depends on `B` and `C`,\n but `B` and `C` both depend on `A` - which version of `A` does `D` get?). Libs\n in traditional langs operate on shared/global namespaces but since Unison\n relies on hashes instead of names there are no conflicts. Both \"versions\" of A\n can co-exist.\n- **Typed durable storage**. Typically when we persist data we serialize\n it into some other form like bytes, JSON, or SQL then write a ton of code to\n transfer in and out of that format and validate it back into some type, e.g.\n `Employee`. When you stop referring to things by name, multiple \"versions\" of\n a type can exist, and you can simply persist and un-persist structures.\n- 🤯 **Programs that deploy themselves** + describe whole elastic distributed\n systems 🤯\n - Because code is content-addressable it's easy to distribute definitions\n across a cluster via nodes asking other nodes for the definition on demand\n when that node is asked to evaluate a function\n - Nodes require no setup to run your code other than provisioning Unison\n - Code can provision new nodes to run code at runtime\n - Think of it as \"distributed programming as a library\"\n - Abilities like `Remote` (which allows code to run on a remote machine) can\n be defined in the type system (this looks like a Type Class).\n - Can define custom ability types and a corresponding interpreter that encodes\n the meaning of that type i.e. what it does at runtime\n - e.g. `Cloud.usEast.run '(dsort hugeDataset)`\n - What about stateful elastic services?\n - Unison is researching this currently. For example: a system with\n high-concurrency that still involves some sort of consensus, like a KV\n store with updates happening concurrently on many nodes\n\nHighly recommend watching the talk.\n\nLearn more at:\n\n- [unisonweb.org](https://unisonweb.org)\n- [Paul's Unison blog posts](https://pchiusano.github.io/unison/)\n","data":{"layout":"article","title":"Notes on Unison\n","categories":["plt","unison","talks","notes"],"comments":true,"excerpt":"Notes on the Strange Loop talk 'Unison: a new distributed programming language'\nby Paul Chiusano. Warning: mind expanding future tech 🤯\n","image":{"feature":"unison_feature.png","teaser":"unison_teaser.png"}},"dateString":"Tue Sep 17 2019","slug":"unison-talk-at-strangeloop","filePath":"2019-09-17-unison-talk-at-strangeloop.mdx"},{"content":"\nMany software engineers are unfamiliar with open source software (OSS) and have\nnever contributed, but we all consume it, heavily. I suggest all software\nengineers have familiarity with OSS and consider contribution as part of the\nfoundational culture and rhythm of software engineering. It could be authoring a\nnew project or contributing to other projects. It could be writing tests and\ndocumentation, or helping users in community chat rooms, or blogging about your\nexperience with various tools and libraries.\n\n## Essentiality\n\nImagine if we didn't have languages, tools and web frameworks to build on top\nof. Or if we had to purchase a license to use some stack (a well known company\nalready tried this and apparently decided [open source was a superior\nstrategy](https://opensource.com/business/14/11/microsoft-dot-net-empower-open-source-communities)).\n\nProgress would be significantly slowed. Instead of going from idea to `rails new` and having a proof of concept in a few hours or days you'd be reinventing\nthe wheel for the nth time or scouring pricing information to figure out which\nstack strikes the right balance of time vs cost savings. In other words: not\nbuilding.\n\n## It legitimizes your work\n\nCode and systems that have gone through the crucible of public scrutiny and\nsurvived are that much stronger, more legitimate, with vastly more permanence.\n\nIt forces the author to create and maintain all the peripheral work around a\nsolid piece of technology. It pushes the tech through edge cases and unexpected\nuse cases. It forces one to write documentation, maintain test coverage, publish\nartifacts, build community, and consider how much generality is sufficient to\nelevate a system from one-off usage to becoming useful in other contexts making\nitself resilient to bit-rot.\n\nIt forces the author to achieve a level of excellence that probably wouldn't\nhave otherwise been realized without the extrinsic motivation.\n\nContrast this with internal code. Whether that code lives or gets extirpated\ndepends on a ton of variable factors in the company. Reorganizations, rewrites,\npolitics, strategy, and all kinds of other factors out of your control affect\nthe longevity of your code. And if it does live on for some amount of time, it\nwill most likely be just good enough to maybe barely do what it's supposed to,\nin a perpetual alpha state. Many Project Managers don't understand the value in\ncomprehensive technical excellence, and therefore won't prioritize technical\ndebt cleanup, necessary refactoring, coverage, documentation, or giving talks.\n\nAfter working X years at Y corp don't you want something to show for yourself?\nAs for me, every time I solve a problem the first thing I ask is \"can this be\nopen sourced?\"\n\n## It sharpens your skills\n\nWhen you put your work out in the open, you invite criticism. That may seem\nscary, but you know makes it more than worth it? Leveling up your skills and\nsharpening your thoughts in the crucible of public opinion. Do hard things and\nyou can't help but grow and become stronger and more mature in the process.\n\n## Social responsibility\n\nGreat. You've decided open source is worth pursuing. You want to make the world\na better place. So what is your responsibility to the community when you open\nsource a project and that project achieves some degree of fame or number of\nusers? Do you \"owe\" your users nothing? Or something?\n\nIf your thing is worth the attention of the community there is a certain\nstandard of excellence that should be met and maintained. By achieving some\nlevel of recognition, whether you wanted it or not, you now have a\nresponsibility to your users.\n\n- make a best effort to ensure the quality and security of your code\n- publish a roadmap\n- let users know how they can contribute\n - optional: welcome new users and offer to mentor engineers who are making\n an honest effort to learn and grow\n- be open to feedback and criticism\n- aim for the good of the community\n- fix bugs in a reasonable amount of time\n- delegate stewardship when appropriate to reduce becoming a single point of\n knowledge\n- pass off stewardship and leadership if you personally can't commit to the\n above\n\nThese are not easy things to do. Ideally we engineers have the support of our\nemployers to uphold this commitment to social good and technical excellence.\nAfter all, it's often the tech companies who are directly benefiting financially\nfrom the wealth of open source technology that they build upon.\n\n## Epilogue\n\nWe consume millions (billions?) of hours of work put into the technology we use\nand built atop every day. Each of us will never come close to giving back as\nmuch as we consume. But please contribute if you can, however great or small,\nfrom a place of joy, curiosity, and sincerity, striving to to advance technology\nand shape community while growing yourself. Be an advocate for building culture\nthat cares about open source at your company.\n\nRaise the floor.\n","data":{"layout":"article","title":"Essentiality of Open Source","categories":["oss","culture"],"comments":true,"excerpt":"Open source is the foundation we build upon.\n"},"dateString":"Tue Apr 7 2020","slug":"essentiality-of-open-source","filePath":"2020-04-07-essentiality-of-open-source.mdx"},{"content":"\nStarting a new job is hard. Really hard. It might take anywhere from 6 months to\n2 years to really hit one's stride.\n\nDepending on the organization, pockets of legacy tech, turnover, culture of\ntechnical excellence (or not) and a hundred other reasons, the state of tech\ncould be anywhere on a spectrum of terrible to amazing. And most likely it\nleans a little harder toward terrible.\n\nThere are many things competing for an IC's time:\n\n- Interviewing\n- Reviewing PRs\n- Meetings\n - Eng allhands\n - Team standups\n - Org standups\n - Company days\n - Demo days\n- and on and on and on\n\nInternal projects tend to have a perpetually-alpha quality. I've seen it at\nevery company I've worked for. It's rare to escape the sticky gravity well of\nperma-alpha. (One trick for doing it is to [open\nsource](/essentiality-of-open-source) your thing, when appropriate).\n\nTo combat this, as software engineers we should leave things in a better state\nevery time we interact with them, even if we don't technically own the thing.\n\nFor example, if I need to make a change to an existing codebase, I hope and pray\nthat I can clone the repo and very quickly find:\n\n1. links to the environments it's running in\n1. how to build (this should be 1 command that works on everyone's machine)\n1. docs (even some basic pointers)\n1. specs for expected shapes of data - hopefully you use a typed language\n1. how to run the tests\n1. a link to the project's CI/CD\n1. etc...\n\nAs an engineer, I want to interact with well-defined black boxes. Until I have a\nreason to understand the internals and turn the black box into a white box by\nconsuming available materials, including the code itself, which is the ultimate\nsource of truth, as well as the other hooks the authors have given me.\n\nA system is a composition of black boxes, each with its own unique\ncharacteristics.\n\nA great engineer should be taking the apps and services they work on beyond\nalpha.\n\nA great SRE or platform architect should be applying the same holistic\nexcellence to the collection of apps they care for. How are you exposing a\nconsistent interface into the system? Is it a GraphQL BFF? A consistent\nadherence to gRPC with well-defined Protobufs?\n\nLeave it better.\n","data":{"layout":"article","title":"Leave it better","categories":["eng-practices"],"comments":true,"excerpt":"Great engineers improve everything they touch.\n"},"dateString":"Thu Apr 22 2021","slug":"leave-it-better","filePath":"2021-04-22-leave-it-better.mdx"},{"content":"\nModern web apps are built in a complex style, requiring many tools\nand systems to support both their development (e.g. CI/CD, secrets, config,\nfeature flags, etc.) and production runtime (e.g. databases, caches, CDNs, etc.).\n\nFor each of these components engineers must decide where on the spectrum of\nfully-managed to custom-built the solution should lie. It can be useful\nto think through the lens of _entropy_ and _extropy_.\n\n## Forces of extropy\n\nDiscounting pivots and shutdowns, the core product itself is undergoing constant\nextropy. Engineers come and go but the product is always on a growth trajectory,\nbeing reconciled from current state into some better state with better utility\nand better design and newer underlying technology. It's the focus of everyone\nfrom CEO to legal to customer support to engineers to designers. High degree of\nattention from a diverse set of disciplines prevents the product from\ndefaulting into a state of entropy.\n\nUnderneath the product are many layers of supporting infrastructure and\ntechnology. The further away from the core product a component lives the less\noften forces of extropy are applied, and the further that component's\nimplementation should be toward the fully-managed side of the spectrum.\n\n## Fast forward\n\nAsk ChatGPT to write feedback 3 years in the future for what you're building,\nwritten by engineers who will join the company 18 months from now. Which of\nthese will it most closely resemble?\n\n\u003e Wow whoever wrote this did a great job thinking about the future needs of\n\u003e the company and set us up for success\n\n\u003e I understand why they did it this way at the time but now it no longer makes\n\u003e sense\n\n\u003e I wish they had integrated this in an orthogonal manner so it'd more of a\n\u003e two-way decision\n\n## Focus\n\nIt might be fun to custom build a CI/CD system, but unless your core product is\nin the CI/CD space, this creates unnecessary, compounding risk and complexity.\nBy focusing on what unique value your product delivers you can build a better\nproduct, faster.\n\nBeware entropy over time.\n\n## Further reading\n\n- [Extropy by Kevin Kelly](https://kk.org/thetechnium/extropy/)\n","data":{"layout":"article","title":"Will it survive entropy?","categories":["infrastructure"],"comments":true,"excerpt":"For every component that makes up a modern web app engineers must decide\nwhere on the spectrum of fully-managed to custom-built the solution should\nlie. It can be useful to think through the lens of entropy and extropy.\n"},"dateString":"Wed May 3 2023","slug":"survive-entropy","filePath":"2023-05-03-survive-entropy.mdx"},{"content":"\nMonorepos have become popular in recent years. What problems are they solving,\nand what problems are they introducing? Should we switch to monorepo?\n\n## Monorepos introduce problems\n\nMonorepos introduce many technical, organizational and political problems.\n\nMonorepos are typically an alternative to standard build tooling. That means\nevery feature your favorite, wildly popular, community-supported build tool's\nfeatures will likely be missing in your monorepo tooling. You either need to\ncustom build it, or tell your users \"sorry, we don't support that\".\n\n## Monorepos increase coupling\n\nIn a large company there are people, teams, services, languages, frameworks that\nexist without you actually knowing about them. This is a good thing. We need\nboundaries and abstractions in order to make the world smaller.\n\nCounterpoint: Google. (Counterpoint: you are not Google).\n\n## Single language vs multi language\n\nThere's a very big difference between monorepo tools that support multiple\nlanguages (like Bazel) and single language monorepo tools like Turborepo, Rush,\nor Lerna.\n\nSingle language tools typically have a much better developer experience and are\nless complex because the language gets first class support. All the tooling is\nbuilt around that particular language, which means there's likely much less of a\ngap between the full ecosystem of tooling for that language and what's available\nin the constrained monorepo world.\n\nContrast that with a tool like Bazel. Bazel is very extensible, so even if your\nlanguage isn't already supported you can add support pretty easily. But\n\"support\" just means opting in to the Bazel way of doing things, including\nits impressive incremental builds and extreme use of caching. What you don't get\nis IDE or standard tooling support. Want to use LSP and NeoVim? Too bad. The\ntools are setup in a way that expect common build tooling (e.g. deps in\n`node_modules` or compiled code in `target`). Bazel breaks all these\nconventions, so the vast ecosystem of tools built upon them just don't work\nunless you go to great lengths (and introduce incredible complexity) to bridge\nthe gap between Bazel-land and established standards.\n\n## Go all in\n\nIf you're going to use a monorepo, you need to absolutely go all in. You need a\ndedicated team to maintain your monorepo indefinitely. **Monorepos are an\nincredibly complex and expensive endeavor** but most people don't talk about\nthis.\n\nThis team should be come an expert in the tooling, whether it's Bazel, Pants,\nBuck, Earthly, Lerna, or something else.\n\nThe monorepo should be the de facto way of building software at your company. Do\nnot end up in a state where you still have polyrepos alongside your monorepo, or\nworse yet: multiple monorepos. This is the path to destruction.\n\n## Vendors as outsourced experts\n\nBecause monorepos are incredibly complex and the tooling is nowhere near on-par\nwith the standard tooling, whole companies have spun up around the monorepo\necosystem. You can hire them to be your experts and help fill the missing\ntooling gaps.\n\nThey are expensive, but they are probably worth it when you compare their fee to\na few annual developer salaries.\n\n**Monorepos are very expensive.** Make sure you're all in and understand the\ncosts before embarking.\n\n## So what's the point?\n\nIt's not all bad. Monorepos exist for a reason, and as long as you can avoid the\nmany traps, it can provide a great developer experience.\n\nMonorepos give you:\n\n- Atomic code changes that you can be developed, tested, reviewed, and released\n together.\n- Managed tooling. The team who supports the monorepo can take care of details\n for you so you no longer need to care about them. Things like security\n patches, dependency upgrades, CI/CD pipelines, and even automatic provisioning\n of infrastructure (although these are all things you can do in a polyrepo\n too).\n- A constrained set of standard langauges, frameworks, dependencies, and\n tooling.\n","data":{"layout":"article","title":"You might not want a Monorepo","categories":["infrastructure"],"comments":true,"excerpt":"Monorepos have become popular in recent years. What problems are they\nsolving, and what problems are they introducing?\n"},"dateString":"Wed Apr 17 2024","slug":"you-might-not-want-a-monorepo","filePath":"2024-04-17-you-might-not-want-a-monorepo.mdx"},{"content":"\nRemember how for decades we researched and iterated a million ways to make\nprogramming languages expressive and correct and built massive ecosystems of\nlibraries, tooling, testing, type checking around them?\n\nThen we throw that out and replace it with 1000 line dumb yamls when it comes to\nCI and cloud infrastructure in general.\n\nThe reasoning goes something like this:\n\n\u003e Let's create a declarative spec that limits what we can do and promotes\n\u003e hermetic deterministic pipelines\n\nbut then you (we) realize we need to do programmy stuff, so:\n\n\u003e Let's make it Turing complete but extremely awkward to express basic\n\u003e conditionals, control flows, parallelism, and dependencies\n\nLook at the iteration of the HashiCorp Configuration Language (HCL), one of the\nworst methods of expression to ever curse SREs. It's declarative but it's also\nimperative. It's the worst of all worlds. How does it work? No one knows.\n\nYAML was a bad idea for CI pipelines in the same way that HCL was a bad idea for\nTerraform. CDK did it much better. (I can't believe I'm saying AWS tech is\nbetter than something).\n\nShould have just used a real programming language.\n","data":{"layout":"article","title":"YAML was a bad idea for CI pipelines","categories":["infrastructure"],"comments":true,"excerpt":"YAML was a bad idea for CI pipelines\n"},"dateString":"Mon Apr 14 2025","slug":"dumb-yaml-ci","filePath":"2025-04-14-dumb-yaml-ci.mdx"}]},"__N_SSG":true},"page":"/","query":{},"buildId":"txPjj0UlODJm11FYMEzVo","isFallback":false,"gsp":true,"scriptLoader":[]}</script></body></html>
Whois info of domain
Domain Name: DEVTH.COM
Registry Domain ID: 956925049_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.cloudflare.com
Registrar URL: http://www.cloudflare.com
Updated Date: 2025-04-03T04:04:43Z
Creation Date: 2007-05-03T23:36:36Z
Registry Expiry Date: 2026-05-03T23:36:36Z
Registrar: Cloudflare, Inc.
Registrar IANA ID: 1910
Registrar Abuse Contact Email: [email protected]
Registrar Abuse Contact Phone: +1.6503198930
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Name Server: AMBER.NS.CLOUDFLARE.COM
Name Server: GERALD.NS.CLOUDFLARE.COM
DNSSEC: signedDelegation
DNSSEC DS Data: 2371 13 2 02E0E64E69F5EF202BBA4BFB780751001BCC9E60838B717806ACABAB0CA48CB7
URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2025-04-30T08:19:23Z <<<
For more information on Whois status codes, please visit https://icann.org/epp
NOTICE: The expiration date displayed in this record is the date the
TERMS OF USE: You are not authorized to access or query our Whois
by the following terms of use: You agree that you may use this Data only
to: (1) allow, enable, or otherwise support the transmission of mass