summary refs log tree commit diff stats
path: root/compiler/modulegraphs.nim
Commit message (Collapse)AuthorAgeFilesLines
* Compiler plugin for implementing incremental computation in user space (#10819)cooldome2019-04-111-1/+51
| | | | | | | | | | This plugin provides essential building block for implementing incremental computations in your programs. The idea behind incremental computations is that if you do the same calculation multiple times but with slightly different inputs you don't have to recompute everything from scratch. Also you don't want to adopt special algorithms either, you would like to write your code in standard from scratch manner and get incrementality for free when it is possible. The plugin computes the digest of the proc bodies, recursively hashing all called procs as well . Such digest with the digest of the argument values gives a good "name" for the result. Terminology loosely follows paper "Incremental Computation with Names" link below. It works well if you have no side effects in your computations. If you have global state in your computations then you will need problem specific workarounds to represent global state in set of "names" . SideEffect tracking in Nim also useful in this topic. Classical examples: Dashboard with ticking data. New data arrives non stop and you would like to update the dashboard recomputing only changed outputs. Excel spreadsheet where user changes one cell and you would like to recompute all cells that are affected by the change, but do not want to recompute every cell in the spreadsheet.
* newruntime: progressAraq2019-04-011-1/+3
|
* Initial version of the hot-code reloading support for native targets (#10729)zah2019-02-261-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * squashed work by Zahary * squashing a ton of useful history... otherwise rebasing on top of upstream Nim after commit 82c009a2cbc5d07ab9a847f1c58228a20efaf219 would be impossible. * Code review changes; Working test suite (without code reloading enabled) * - documentation - implemented the HCR test - almost works... - fix the issue on Unix where for executable targets the source file for the main module of a project in nimcache was being overwritten with the binary itself (and thus the actual source code was lost) - fixing embedded paths to shared objects on unix (the "lib" prefix was being prepended to the entire path instead of just the filename) - other fixes - removing unnecessary includes since that file is already included in chcks.nim which is in turn included in system.nim (and previously was getting imported in chcks.nim but then system.nim improts something... and that breaks HCR (perhaps it could be fixed but it would be nice not to import anything in system)) * fix for clang & C++ - explicitly casting a function pointer to void* more stable mangling of parameter names when HCR is on the length of the static arrays in the DatInit functions is now part of the name of the variables, so when they get resized they get also recreated more stable mangling for inline functions - no longer depends on the module which first used them work on the new complicated HCR test - turned surprisingly complex - WIP test now successfully passes even when re-running `koch test` (previously when the nimcache wasn't cold that lead to errors) better documentation calling setStackBottomWith for PreMain passes over the HcrInit/DatInit/Init calls of all modules are now in the proper order (first all of one type, then all of the next). Also typeinfo globals are registered (created) in a single pass before the DatInit pass (because of the way generic instantiations are handled) Fix the test suite execution on macOs fix for being able to query the program arguments when using HCR on posix! other fixes * Bugfix: Fix a compilation error in C++ mode when a function pointer is converted to a raw pointer * basic documentation for the new hot code reloading semantics * Add change log entry * Don't re-execute the top-level statements while reloading JS code * fix a number of tests broken in a recent bugfix * Review changes * Added {.executeOnReload.} pragma that indicates top-level statements that should be executed on each reload. To make this work, I've modified the way the `if (hcr_init_) {...}` guards are produced in the init code. This still needs more work as the new guards seem to be inserted within the previously generated guards. This change also removes the need for `lastRegistedGlobal` in nimhcr. * Implemented the `signatureHash` magic and the `hasModuleChanged` API depending on it (the actual logic is not imlemented yet). * Add the "hcr" prefix to all HCR-related symbols in the system module. Added a new `hotcodereloading` module exporting the high-level API to the user. Besides being more hygienic, this was also required in order to make it possible to use macros in the high-level API. Without the split, `system` would have to import `macros`, which was going to produce the well-known init problems. * Attempted to solve the "GC markers problem". Crashes were expected with the previous code, because the GC markers were compiled as normal procs are registered in the GC. When their module is unloaded, dangling pointers will remain in the GC tables. To solve this issue, I don't register any GC markers when HCR is on, but I add them to the HCR globals metadata and I use a single marker registed in nimhcr during the initialization of the system module that will be responsible for marking all globals. * fix a compilation error * - implemented the hasModuleChanged functionality - tuples can be returned and broken into different vars in global scope - added comments for the closnig scopes of the if statements in the init proc - the new executeOnReload pragma works now! - other fixes * finally! fixing this hack in a proper way - declaring the destructor out of line (out of the class body) - we no longer need to forward-declare popCurrentExceptionEx * Force full module parsing This is a temporary hack that breaks some tests. I'll investigate later how these can be fixed. * tuples are now properly handled when global! * these comments mess up the codegen in debug mode when $n is not actually a new line (or something like that) - these labels are intended only for GOTO labels anyway... * "solved" the issue with the .pdb locks on windows when a binary is being debugged and hot code reloading is used at the same time * fixes after rebasing... * small fixes for the test * better handling of globals! no more compiler crashes for locals with the global pragma, also simplified code around loops in global scope which have local vars (actually globals) * we can now use the global pragma even for ... globals! * the right output * lets try those boehm GC tests * after the test is ran it will be at its starting state - no git modifications * clarification in the docs * removed unnecessary line directives for forward declarations of functions - they were causing trouble with hot code reloading when no semantic change propagates to the main module but a line directive got changed and thus the main module had to be recompiled since the .c code had changed * fixed bug! was inserting duplicate keys into the table and later was removing only 1 copy of all the duplicates (after a few reloads) * no longer breaking into DatInit code when not supposed to * fixes after rebasing * yet more fixes after rebasing * Update jssys.nim * Rework the HCR path-handling logic After reviewing the code more carefully, I've noticed that the old logic will be broken when the user overrides the '--out:f' compiler option. Besides fixing this issues, I took the opportunity to implement the missing '--outdir:d' option. Other changes: * ./koch test won't overwrite any HCR and RTL builds located in nim/lib * HCR and RTL are compiled with --threads:on by default * Clean up the globals registration logic * Handle non-flattened top-level stmtlists in JS as well * The HCR is not supported with the Boehm GC yet Also fixes some typos and the expected output of the HCR integration test * The GC marker procs are now properly used as trampolines * Fix the HCR integration test in release builds * Fix ./koch tools * this forward declaration doesn't seem to be necessary, and in fact breaks HCR because a 2nd function pointer is emitted for this externed/rtl func * the forward declaration I removed in the last commit was actually necessary * Attempt to make all tests green * Fix tgenscript * BAT file for running the HCR integration test on Windows [skip ci] * Fix the docgen tests * A final fix for Travis (hopefully)
* compiler refactoring: TSymSeq is gone for goodAraq2018-12-111-1/+1
|
* added first version of a nimfind tool for the poor souls that don't have a ↵Andreas Rumpf2018-11-141-0/+29
| | | | good nimsuggest integretation
* Make the registered passes local to the ModuleGraph (#9259)LemonBoy2018-10-091-0/+13
| | | Closes #9068
* deprecate the .this pragmaAraq2018-08-091-28/+26
|
* completed VM support for incremental compilationsAndreas Rumpf2018-06-041-1/+7
|
* incremental compilation: implemented basic replay logicAndreas Rumpf2018-06-021-0/+5
|
* document how the incremental compilation scheme could workAndreas Rumpf2018-06-011-1/+1
|
* refactoring: move DB model to incremental.nimAndreas Rumpf2018-05-301-4/+7
|
* vm now free of global variablesAndreas Rumpf2018-05-281-0/+2
|
* refactoring: remove idents.legacy global variable and pass the IdentCache ↵Andreas Rumpf2018-05-271-7/+6
| | | | around explicitly
* remove more global variables in the Nim compilerAndreas Rumpf2018-05-271-1/+1
|
* remove ast.emptyNode global; cleanup configuration.nimAraq2018-05-161-0/+2
|
* options.nim: no global variables anymoreAndreas Rumpf2018-05-131-1/+1
|
* sem pass compiles againAndreas Rumpf2018-05-121-1/+8
|
* big refactoring: magicsys compiles againAndreas Rumpf2018-05-101-0/+9
|
* refactoring: make FileIndex a distinct type; make line information an ↵Andreas Rumpf2018-04-211-19/+21
| | | | uint16; fixes #7654
* symbol files: implemented accurate module dependency trackingAraq2018-02-211-1/+4
|
* big compiler refactoring; avoid globals for multi method dispatcher generationAndreas Rumpf2017-02-221-0/+8
|
* nimsuggest uses multithreading and full project recompilesAndreas Rumpf2017-02-131-0/+4
|
* disable transitive closure computations; only semcheck procs that have the ↵Andreas Rumpf2017-02-091-1/+1
| | | | cursor
* attempt to make nimble work againAndreas Rumpf2017-02-021-1/+4
|
* refactoring: explict config state instead of globalsAndreas Rumpf2017-02-011-2/+4
|
* happy new yearAraq2017-01-071-1/+1
|
* refactoring: C codegen with fewer global variablesAraq2016-12-051-0/+1
|
* disallow recursive module dependenciesAraq2016-11-231-0/+4
|
* nimsuggest supports include files properly; added the compiler itself as a ↵Andreas Rumpf2016-11-061-1/+17
| | | | testcase
* added missing moduleAraq2016-11-051-0/+92
2016-12-26 11:44:14 -0800 committer Kartik K. Agaram <vc@akkartik.com> 2016-12-26 11:44:14 -0800 3710' href='/akkartik/mu/commit/html/033exclusive_container.cc.html?h=main&id=204dae921abff0c70e017215bb3c91fa6ca11aff'>204dae92 ^
5fe060d5 ^

104e521c ^
805d58c6 ^
204dae92 ^
5fe060d5 ^


e4ac3c9e ^
5fe060d5 ^









e4ac3c9e ^
5fe060d5 ^








e4ac3c9e ^
5fe060d5 ^
104e521c ^
e4ac3c9e ^
5fe060d5 ^


e4ac3c9e ^
104e521c ^
e4ac3c9e ^
5fe060d5 ^



104e521c ^
e4ac3c9e ^
5fe060d5 ^











e4ac3c9e ^


5fe060d5 ^























































4945e77a ^
5fe060d5 ^









4945e77a ^
5fe060d5 ^


























4945e77a ^
5fe060d5 ^

4945e77a ^
5fe060d5 ^


























104e521c ^
5fe060d5 ^



e4ac3c9e ^
5fe060d5 ^




e4ac3c9e ^
5fe060d5 ^


104e521c ^
5fe060d5 ^

























































4945e77a ^
5fe060d5 ^
4945e77a ^
5fe060d5 ^









































e4ac3c9e ^
5fe060d5 ^






e4ac3c9e ^
5fe060d5 ^























672e3e50 ^


a654e4ec ^
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526