thought/write-posix-shell.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
#!/bin/sh what is shell? shell is a language. but most people in tech are familiar with shell scripts. ... shit. actually. wait, let me rephrase- most people in tech _use_ shell scripts. most people in tech _hate_ shell scripts. most people in tech _avoid_ shell scripts. and for good reason. because people hate writing shell (the language). why do people hate shell? arcane syntax shell's syntax is cumbersome and arcane because it was invented around the time people thought lawn darts were a good idea. confusing error handling in python there's try/except. in golang, errors are objects. in shell, errors just sort of happen in insane ways. hard to maintain when shell grows beyond a few hundred lines or so, it becomes impossible to maintain. unique and weird since shell has no "canonical form" shell scripts can look wildly different depending on the author, which furthers confusion. perception in the tech industry, people often look down on shell scripts as tech debt / lame, and devs are often incentiviced to use hotter, more exciting things. so why use shell at all? simply: because shell is an insanely productive language. in fact, i believe that shell is the *most* productive language. in terms of time + brainpower spent to produce a result, shell can do in 10 seconds what would take much longer in any other language, because shell is optimized for speed, instant feedback, and is enhanced by many years of development. it's also ubiquitous, and accessible almost anywhere you need it. people often think that they're not capable of producing things as quickly as i can. but here's the thing: i'm dumb as hell. i just know a little bit of shell. <ascii> bash me in the head like human languages, shell has many dialects. there's the common bash and zsh shell dialects. there's fish. there's ksh. dash. csh. mrsh. the list goes on. and on. and on. each dialect has varying levels of compatability. you might use a the bash dialect of shell, but struggle to run your scripts in your alpine container. you might write a script on macos, only to realize that it won't run on linux. when people write shell, they will often say things like "i'm writing bash" or "i'm bash scripting" this is because bash is the most common shell dialect. i want to be clear: if you "write bash", you're really just writing shell with some bash features. if you happen to not use any bash features, then you're just writing plain shell. the reason i'm being pedantic about "bash" is because i believe that most people should only care about a single dialect of shell: posix shell this opinion is not very widespread, but i think that it should be. let me explain ~eternal sunshine of the posix shell* posix shell is simple. it is shell written by following the posix specification. the posix specification is hosted on the worst website i've ever seen[1], and i partially blame that website for posix shell being unpopular. if you write posix shell, you're in for some benefits. posix shell is compatible: it'll run on debian, on openbsd, an alpine container, macos! illumos! even fucking AIX! posix shell is defined: the posix spec fully defines every shell command it allows the defined commands have defined behavior and have specific flags that do specific things posix shell is pretty small: if you like reading specs for fun (aka if you like torture), you might read through the posix spec - it'd only take a day or two. of, if you're not a massochist, just read the parts of it that apply to the problems you solve, and piece it together over time. what feels bad: running arcane shell commands from stackoverflow, having them work, and being like "uhhhh wut.jpg" and moving on because "shell is arcane magicks" what feels good: running posix shell commands that you reference from the specification, knowing that they'll simply work everwhere for eternity. posix shell is eternal: imagine posix shell as the "standard library of shell" the concrete foundation on which you stand. /------------------------------\ | | | omg! | | posix shell is amazing! | | i wanna use it everywhere! | | i wanna go use it RIGHT NOW! | \------------------------------/* * * (you, probably) 🤔 slow down there you son of a gun!! yes, posix shell is amazing, but it's also very constraining. consider this sentence: "arrays do not exist in posix shell" ... "NOOO!" you shout, because you like data structures a LOT "now THAT'S a good reason to use bash! bash supports arrays!" absolutely not! i believe that you should use shell if your problem is: - small and scoped (~200 lines of shell or less) - unlikely to increase in size and scope as time goes on - not very complex here's some simple rules to follow: 1. if you suspect that a shell script could be more than ~200 lines, use a programming language instead. 2. if your shell script needs arrays, structs, maps, or any complex data structure, you have officially breached the shell complexity ceiling and should stop right now. use a real programming language instead. 3. if you need to do complex error handling or if the program must be very reliable or pro posix shell tips from j3s if you have a suspicion that the scope of your shell script may need to grow over time, use a real programming language instead. use #!/bin/sh at the top do not use set -e unless you have good reason every command you use that is not in the posix spec you should list as a dependency. implement a die() function keep your pipe chains short posix shell is all you need be careful posix shell posix shell is different. posix shell is the best idea with the worst website i've ever seen.[1] the URL is horrific. the display is horrific. but the content is golden. i often search for "posix shell <command>" and hunt for the opengroup.org URL - it my secret is shell. my "superpower" is shell. but not regular shell. can whip up a script in ten seconds that downloads a file, formats its output, writes it to another file in the correct format. unfortunately, - maintenance: maintaining shell scripts is a nightmare - bash opinion: as a tech community, we need to write more posix shell. but not for the reasons you're imagining. for inspiration, check out how russ cox uses shell to make decisions about golang's development. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html sidenote: if anyone out there wants to write a posix shell website that doesn't suck absolute ass, let me know.