Common Lisp / Scheme macros

Here is a collection of snippets I’ve found helpful for understanding Common Lisp and Scheme macros …

 

“Macros are the single greatest advantage that Lisp has as a programming language and the single greatest advantage of any programming language. With them you can do things that you simply cannot do in other languages. Because macros can be used to transform lisp into other programming languages and back, programmers who gain experience with them discover that all other languages are just skins on top of lisp. This is the big deal. Lisp is special because programming with it is actually programing at a higher level. Where most languages invent and enforce syntactic and semantic rules, lisp is general and malleable. With lisp, you make the rules. […]

[…] macros are still not used to the fullest possible extent by most lisp programmers and are not used at all by all other programmers. This has always been a conundrum for advanced lispers. Since macros are so great, why doesn’t everybody use them all the time? While it’s true that the smartest, most determined programmers always end up at lisp macros, few start their programming careers there. Understanding why macros are so great requires understanding what lisp has that other languages don’t. It requires an understanding of other, less powerful languages. Sadly, most programmers lose the will to learn after they have mastered a few other languages and never make it close to understanding what a macro is or how to take advantage of one. But the top percentile of programmers in any language are always forced to learn some sort of way to write programs that write programs: macros. Because it is the best language for writing macros, the smartest and most determined and most curious programmers always end up at lisp.

Although the top-percentile of programmers is necessarily a small number, as the overall programming population grows so does the number of top-percentile programmers. The programming world sees few examples of the power of macros and understands far fewer, but this is changing. Because of the productivity multiplication that can be achieved through macros, the age of the macro is coming, whether the world is ready or not. […] Be prepared.

The conventional wisdom surrounding macros is to use them only when necessary because they can be difficult to understand, contain extremely subtle bugs, and limit you in possibly surprising ways if you think of everything as functions. These aren’t defects in the lisp macro system itself but instead are traits of macro programming in general. As with any technology, the more powerful the tool, the more ways there are to misuse it. And, as far as programming constructs go, lisp macros are the most powerful tool.

An interesting parallel to learning macros in lisp is that of learning pointers in the C programming language. Most beginning C programmers are able to quickly pick up most of the language. Functions, types, variables, arithmetic expressions: all have parallels in previous intellectual experiences beginners might have had, from elementary school maths to experimenting with simpler programming languages. But most novice C programmers hit a brick wall when they encounter pointers.

Pointers are second nature to experienced C programmers, most of whom consider their complete understanding necessary for the proper use of C. Because pointers are so fundamental, most experienced C programmers would not advise limits on their use for stylistic or learning purposes. Despite this, many C novices feel pointers are an unnecessary complication and avoid their use, resulting in the FORTRAN in any language symptom where valuable language features are neglected. The disease is ignorance of the language’s features, not poor programming style. Once the features are fully understood, the correct styles are obvious. An auxiliary theme […] is that in programming, style is not something to pursue directly. Style is necessary only where understanding is missing1.

Like C pointers, the macro is a feature of lisp that is often poorly understood, the wisdom on its proper use being very distributed and idealised. If when considering macros you find yourself relying on stylistic aphorisms like

Macros change the syntax of lisp code.

Macros work on the parse tree of your program.

Only use macros when a function won’t do.

you are probably missing the big picture when it comes to macro programming.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Introduction)


“When writing computer programs, certain patterns arise over and over again. For example, programs must often loop through the elements of arrays, increment or decrement the values of variables, and perform multi-way conditionals based on numeric or character values. Programming language designers typically acknowledge this fact by including special-purpose syntactic constructs that handle the most common patterns.  C, for instance, provides multiple looping constructs, multiple conditional constructs, and multiple constructs for incrementing or otherwise updating the value of a variable.

Some patterns are less common but can occur frequently in a certain class of programs or perhaps just within a single program. These patterns might not even be anticipated by a language’s designers, who in any case would typically choose not to incorporate syntactic constructs to handle such patterns in the language core. Yet, recognizing that such patterns do arise and that special-purpose syntactic constructs can make programs both simpler and easier to read, language designers sometimes include a mechanism for syntactic abstraction , such as C’s preprocessor macros or Common Lisp macros.[…]

Syntactic abstraction facilities differ in several significant ways. C’s preprocessor macros are essentially token-based, allowing the replacement of a macro call with a sequence of tokens with text from the macro call substituted for the macro’s formal parameters, if any. Lisp macros are expression-based, allowing the replacement of a single expression with another expression, computed in Lisp itself and based on the subforms of the macro call, if any.”

— R. Kent Dybvig, Syntactic Abstraction: The syntax-case expander


“A Lisp macro is like a function that takes Lisp forms or objects as input, and typically generates code to be then compiled and executed. This happens before runtime, in a phase called macroexpansion time. Macros can perform arbitrary computation during expansion, using the full Lisp language.

One use of macros is transforming input representing arbitrary source code into a version specified in terms of known definitions. In other words, macros can add new syntax to the original language (this is known as syntactic abstraction).

This enables easily embedding domain-specific languages, since specialized syntax can be added before runtime.

The chief benefit of macros is that they add power by letting the programmer express intent clearly and with less code. Particularly, one can add new features to the language that appear as if they were builtin. In addition, when used to pre-emptively compute data or initialize state, macros may aid in performance optimisation.”

— Abhishek Reddy, Features of Common Lisp

 


“An important principle of macro design is top-down programming. When designing a lisp macro, you want to start with the abstraction first. You want to write programs that use the macro long before you write the macro itself. Somewhat paradoxically, you need to know how to program in a language before you can write a concise definition/implementation for that language.

So the first step in serious macro construction is always to write use cases of the macro, even though there is no way you can test or use them. If the programs written in the new language are comprehensive enough, a good idea of what will be required to implement a compiler or interpreter for the language will follow.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Chapter 5: Programs that Program)

 

+++

+++


 

“In Common Lisp, the rule is simple. If the first symbol of a form is defined as a macro, the macroexpander is called with the unevaluated arguments of the form, and then the return value is recursively macroexpanded. Since everything’s an S-expression, you don’t need any more than this. Any macro you create will have the same syntax as normal Lisp forms.”

— Random stranger @ C2-Wiki …


///

“Macros can leverage a technique called implicit context. In code that is used frequently, or absolutely must be concise and lacking any surrounding book-keeping cruft, we sometimes choose to implicitly add lisp code around portions of an expression so we don’t have to write it every time we make use of an abstraction. We’ve talked before about implicit contexts and it should be clear that even when not programming macros they are a fundamental part of lisp programming: let and lambda forms have an implicit progn because they evaluate, in sequence, the forms in their body and return the last result.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Chapter 5: Programs that Program)


“In essence, hygienic macro expansion implements lexical scoping with respect to the source code, whereas unhygienic expansion implements lexical scoping with respect to the code after expansion.”

Architectural implications of NAT

“The architectural intent of NAT is to divide the Internet into independent address administrations […] The result of this division is to enforce a client/server architecture (vs. peer/peer) where the servers need to exist in the public address realm.

A significant factor in the success of the Internet is the flexibility derived from a few basic tenets. Foremost is the End- to-End principle, which notes that certain functions can only be performed in the endpoints, thus they are in control of the communication, and the network should be a simple datagram service that moves bits between these points. Restated, the endpoint applications are often the only place capable of correctly managing the data stream. Removing this concern from the lower layer packet-forwarding devices streamlines the forwarding process, contributing to system-wide efficiency.

Another advantage is that the network does not maintain per connection state information. This allows fast rerouting around failures through alternate paths and to better scaling of the overall network. Lack of state also removes any requirement for the network nodes to notify each other as endpoint connections are formed or dropped. Furthermore, the endpoints are not, and need not be, aware of any network components other than the destination, first hop router(s), and an optional name resolution service. Packet integrity is preserved through the network, and transport checksums and any address-dependent security functions are valid end-to-end.

NAT devices (particularly the NAPT variety) undermine most of these basic advantages of the end-to-end model, reducing overall flexibility, while often increasing operational complexity and impeding diagnostic capabilities.”

From IETF RFC 2993: “Architectural Implications of NAT”, November 2000

Jimmy Carter’s Blood-Drenched Legacy

Two Young Girls Found Alongside the Highway to Comalapa Airport Photo: John Hoagland
Two Young Girls Found Alongside the Highway to Comalapa Airport, El Salvador (August 1980) Photo: John Hoagland

“[…] the self-professed advocate for human rights demonstrated quite the penchant for bloodshed. While he didn’t initiate any aggressive invasions of foreign nations the way his predecessors and successors did in Vietnam, Grenada, Panama, Iraq, Afghanistan and many other countries, Carter proved remarkably generous at providing financial, military, diplomatic and ideological support for fascist dictatorships that tortured and killed millions of members of their domestic populations in an effort to crush popular movements for social justice. Some of the regimes he backed carried out mass slaughter that amounted to genocide.

Below are some of Carter’s most shameful and indefensible foreign policy positions that caused monumental levels of death, destruction and suffering for poor, socially disenfranchised people from Asia to Latin America to Africa.” […]

Read the full article at Counterpunch: http://www.counterpunch.org/2015/08/18/jimmy-carters-blood-drenched-legacy/

We Wake The Day

We wake; we wake the day,
the light rising in us like sun–
our breath a prayer brushing
against the feathers in our hands.
We stumble out into streets;
patterns of wire invented by strangers
are strung between eye and sky,
and we dance in two worlds,
inevitable as seasons in one,
exotic curiosities in the other
which rushes headlong down highways,
watches us from car windows, explains
us to its children in words
that no one could ever make
sense of. The image obscures
the vision, and we wonder
whether anyone will ever hear
our own names for the things
we do. Light dances in the body,
surrounds all living things–
even the stones sing
although their songs are infinitely
slower than the ones we learn
from trees. No human voice lasts
long enough to make such music sound.
Earth breath eddies between factories
and office buildings, caresses the surface
of our skin; we go to jobs, the boss
always watching the clock to see
that we’re on time. He tries to shut
out magic and hopes we’ll make
mistakes or disappear. We work
fast and steady and remember
each breath alters the composition
of the air. Change moves relentless,
the pattern unfolding despite their planning–
we’re always there–singing round dance
songs, remembering what supports
our life–impossible to ignore.

-by Gail Tremblay, in Reinventing the Enemy’s Language

Outrage over bumfights vs. porn

Do you remember Bumfights where a bunch of arseholes took a bunch of socio-economically disadvantaged/homeless men and boys and made them do humiliating, exploitative and dangerous acts on film for money/food/etc? Remember how fucking outraged people were?

Now, there’s currently a multi-billion dollar industry doing the exact same thing to countless girls and women and men masturbate to it. And yet, apparently, it’s “free speech”.

— pornographicmeatnightmares on Tumblr

Put Away The Flags (Howard Zinn)

(via The Progressive, 4th July 2006)

US soldier holding flag that says: "There is no flag large enough to cover the shame of killing innocent people."On this July 4, we would do well to renounce nationalism and all its symbols: its flags, its pledges of allegiance, its anthems, its insistence in song that God must single out America to be blessed.

Is not nationalism — that devotion to a flag, an anthem, a boundary so fierce it engenders mass murder — one of the great evils of our time, along with racism, along with religious hatred?

These ways of thinking — cultivated, nurtured, indoctrinated from childhood on — have been useful to those in power, and deadly for those out of power.

National spirit can be benign in a country that is small and lacking both in military power and a hunger for expansion (Switzerland, Norway, Costa Rica and many more). But in a nation like ours — huge, possessing thousands of weapons of mass destruction — what might have been harmless pride becomes an arrogant nationalism dangerous to others and to ourselves.

Our citizenry has been brought up to see our nation as different from others, an exception in the world, uniquely moral, expanding into other lands in order to bring civilization, liberty, democracy.

That self-deception started early.

When the first English settlers moved into Indian land in Massachusetts Bay and were resisted, the violence escalated into war with the Pequot Indians. The killing of Indians was seen as approved by God, the taking of land as commanded by the Bible. The Puritans cited one of the Psalms, which says: “Ask of me, and I shall give thee, the heathen for thine inheritance, and the uttermost parts of the Earth for thy possession.”

When the English set fire to a Pequot village and massacred men, women and children, the Puritan theologian Cotton Mather said: “It was supposed that no less than 600 Pequot souls were brought down to hell that day.”

On the eve of the Mexican War, an American journalist declared it our “Manifest Destiny to overspread the continent allotted by Providence.” After the invasion of Mexico began, The New York Herald announced: “We believe it is a part of our destiny to civilize that beautiful country.”

It was always supposedly for benign purposes that our country went to war.

We invaded Cuba in 1898 to liberate the Cubans, and went to war in the Philippines shortly after, as President McKinley put it, “to civilize and Christianize” the Filipino people.

As our armies were committing massacres in the Philippines (at least 600,000 Filipinos died in a few years of conflict), Elihu Root, our secretary of war, was saying: “The American soldier is different from all other soldiers of all other countries since the war began. He is the advance guard of liberty and justice, of law and order, and of peace and happiness.”

We see in Iraq that our soldiers are not different. They have, perhaps against their better nature, killed thousands of Iraq civilians. And some soldiers have shown themselves capable of brutality, of torture.

Yet they are victims, too, of our government’s lies.

How many times have we heard President Bush tell the troops that if they die, if they return without arms or legs, or blinded, it is for “liberty,” for “democracy”?

One of the effects of nationalist thinking is a loss of a sense of proportion. The killing of 2,300 people at Pearl Harbor becomes the justification for killing 240,000 in Hiroshima and Nagasaki. The killing of 3,000 people on Sept. 11 becomes the justification for killing tens of thousands of people in Afghanistan and Iraq.

And nationalism is given a special virulence when it is said to be blessed by Providence. Today we have a president, invading two countries in four years, who announced on the campaign trail in 2004 that God speaks through him.

We need to refute the idea that our nation is different from, morally superior to, the other imperial powers of world history.

We need to assert our allegiance to the human race, and not to any one nation.

Frantz Fanon: “Concerning Violence”

amazon-logging-2Decolonization is always a violent phenomenon … Decolonization, which sets out to change the order of the world, is, obviously, a program of complete disorder. But it cannot come as a result of magical practices, nor of a natural shock, nor of a friendly understanding. Decolonization, as we know, is a historical process: that is to say that it cannot be understood, it cannot become intelligible nor clear to itself except in the exact measure that we can discern the movements which give it historical form and content. …  If we wish to describe it precisely, we might find it in the wellknown words: “The last shall be first and the first last.” Decolonization is the putting into practice of this sentence. That is why, if we try to describe it, all decolonization is successful.

The naked truth of decolonization evokes for us the searing bullets and bloodstained knives which emanate from it. For if the last shall be first, this will only come to pass after a murderous and decisive struggle between the two protagonists. That affirmed intention to place the last at the head of things, and to make them climb at a pace (too quickly, some say) the well-known steps which characterize an organized society, can only triumph if we use all means to turn the scale, including, of course, that of violence.

You do not turn any society, however primitive it may be, upside down with such a program if you have not decided from the very beginning, that is to say from the actual formulation of that program, to overcome all the obstacles that you will come across in so doing. The native who decides to put the program into practice, and to become its moving force, is ready for violence at all times. From birth it is clear to him that this narrow world, strewn with prohibitions, can only be called in question by absolute violence. …

During the period of decolonization, the natives’ reason is appealed to. He is offered definite values, he is told frequently that decolonization need not mean regression, and that he must put his trust in qualities which are welltried, solid, and highly esteemed. But it so happens that when the native hears a speech about Western culture he pulls out his knife–or at least he makes sure it is within reach. The violence with which the supremacy of white values is affirmed and the aggressiveness which has permeated the victory of these values over the ways of life and of thought of the native mean that, in revenge, the native laughs in mockery when Western values are mentioned in front of him. In the colonial context the settler only ends his work of breaking in the native when the latter admits loudly and intelligibly the supremacy of the white man’s values. In the period of decolonization, the colonized masses mock at these very values, insult them, and vomit them up. …

Baltimore man holding up toilet paper ... looting with police vans burning in background
A man carries items from a store as police vehicles burn, Monday, April 27, 2015, after the funeral of Freddie Gray in Baltimore (AP Photo/Patrick Semansky)

Now what we must never forget is that the immense majority of colonized peoples is oblivious to these problems. For a colonized people the most essential value, because the most concrete, is first and foremost the land: the land which will bring them bread and, above all, dignity. But this dignity has nothing to do with the dignity of the human individual: for that human individual has never heard tell of it. All that the native has seen in his country is that they can freely arrest him, beat him, starve him: and no professor of ethics, no priest has ever come to be beaten in his place, nor to share their bread with him. As far as the native is concerned, morality is very concrete; it is to silence the settler’s defiance, to break his flaunting violence–in a word, to put him out of the picture. The wellknown principle that all men are equal will be illustrated in the colonies from the moment that the native claims that he is the equal of the settler. One step more, and he is ready to fight to be more than the settler. In fact, he has already decided to eject him and to take his place; as we see it, it is a whole material and moral universe which is breaking up. …

Thus the native discovers that his life, his breath, his beating heart are the same as those of the settler. He finds out that the settler’s skin is not of any more value than a native’s skin; and it must be said that this discovery shakes the world in a very necessary manner. All the new, revolutionary assurance of the native stems from it. For if, in fact, my life is worth as much as the settler’s, his glance no longer shrivels me up nor freezes me, and his voice no longer turns me into stone. I am no longer on tenterhooks in his presence; in fact, I don’t give a damn for him. Not only does his presence no longer trouble me, but I am already preparing such efficient ambushes for him that soon there will be no way out but that of flight. …

All the Mediterranean values–the triumph of the human individual, of clarity, and of beauty–become lifeless, colorless knickknacks. All those speeches seem like collections of dead words; those values which seemed to uplift the soul are revealed as worthless, simply because they have nothing to do with the concrete conflict in which the people is engaged. Individualism is the first to disappear. The native intellectual had learnt from his masters that the individual ought to express himself fully. The colonialist bourgeoisie had hammered into the native’s mind the idea of a society of individuals where each person shuts himself up in his own subjectivity, and whose only wealth is individual thought. Now the native who has the opportunity to return to the people during the struggle for freedom will discover the falseness of this theory. The very forms of organization of the struggle will suggest to him a different vocabulary. Brother, sister, friend–these are words outlawed by the colonialist bourgeoisie, because for them my brother is my purse, my friend is part of my scheme for getting on. The native intellectual takes part, in a sort of auto-da-fé, in the destruction of all his idols: egoism, recrimination that springs from pride, and the childish stupidity of those who always want to have the last word. Such a colonized intellectual, dusted over by colonial culture, will in the same way discover the substance of village assemblies, the cohesion of people’s committees, and the extraordinary fruitfulness of local meetings and groupments. Henceforward, the interests of one will be the interests of all, for in concrete fact everyone will be discovered by the troops, everyone will be massacred–or everyone will be saved. …

What are the forces which in the colonial period open up new outlets and engender new aims for the violence of colonized peoples? In the first place there are the political parties and the intellectual or commercial elites. Now, the characteristic feature of certain political structures is that they proclaim abstract principles but refrain from issuing definite commands. The entire action of these nationalist political parties during the colonial period is action of the electoral type: a string of philosophicopolitical dissertations on the themes of the rights of peoples to self-determination, the rights of man to freedom from hunger and human dignity, and the unceasing affirmation of the principle: “One man, one vote.” The national political parties never lay stress upon the necessity of a trial of armed strength, for the good reason that their objective is not the radical overthrowing of the system. Pacifists and legalists, they are in fact partisans of order, the new order–but to the colonialist bourgeoisie they put bluntly enough the demand which to them is the main one: “Give us more power.” On the specific question of violence, the elite are ambiguous. They are violent in their words and reformist in their attitudes. When the nationalist political leaders say something, they make quite clear that they do not really think it.

This characteristic on the part of the nationalist political parties should be interpreted in the light both of the make-up of their leaders and the nature of their followings. The rank-and-file of a nationalist party is urban. The workers, primary schoolteachers, artisans, and small shopkeepers who have begun to profit–at a discount, to be sure–from the colonial setup, have special interests at heart. What this sort of following demands is the betterment of their particular lot: increased salaries, for example. The dialogue between these political parties and colonialism is never broken off. Improvements are discussed, such as full electoral representation, the liberty of the press, and liberty of association. Reforms are debated. Thus it need not astonish anyone to notice that a large number of natives are militant members of the branches of political parties which stem from the mother country. These natives fight under an abstract watchword: “Government by the workers,” and they forget that in their country it should be nationalist watchwords which are first in the field. The native intellectual has clothed his aggressiveness in his barely veiled desire to assimilate himself to the colonial world. He has used his aggressiveness to serve his own individual interests.

Thus there is very easily brought into being a kind of class of affranchised slaves, or slaves who are individually free. What the intellectual demands is the right to multiply the emancipated, and the opportunity to organize a genuine class of emancipated citizens. On the other hand, the mass of the people have no intention of standing by and watching individuals increase their chances of success. What they demand is not the settler’s position of status, but the settler’s place. The immense majority of natives want the settler’s farm. For them, there is no question of entering into competition with the settler. They want to take his place.

The peasantry is systematically disregarded for the most part by the propaganda put out by the nationalist parties. And it is clear that in the colonial countries the peasants alone are revolutionary, for they have nothing to lose and everything to gain. The starving peasant, outside the class system, is the first among the exploited to discover that only violence pays. For him there is no compromise, no possible coming to terms; colonization and decolonization are simply a question of relative strength. The exploited man sees that his liberation implies the use of all means, and that of force first and foremost. When in 1956, after the capitulation of Monsieur Guy Mollet to the settlers in Algeria, the Front de Libération Nationale, in a famous leaflet, stated that colonialism only loosens its hold when the knife is at its throat, no Algerian really found these terms too violent. The leaflet only expressed what every Algerian felt at heart: colonialism is not a thinking machine, nor a body endowed with reasoning faculties. It is violence in its natural state, and it will only yield when confronted with greater violence.

From Frantz Fanon, The Wretched of The Earth

Steven Mintz, Huck’s Raft: Childhood in history, dependence, autonomy, community

“In certain respects, today’s children are more autonomous than young people have ever been. They have their own institutions and media, most now have their own rooms, and many teens have their own cars. Contemporary children mature faster physiologically than those in the past and are more knowledgeable about sexuality, drugs, and other adult realities. They are also more fully integrated into the realm of consumer culture at an earlier age. Yet from the vantage point of history, contemporary children’s lives are more regimented and constrained than ever before. Contemporary society is extreme in the distinction it draws between the worlds of childhood and youth, on the one hand, and of adulthood, on the other. Far more than previous generations, we have prolonged and intensified children’s emotional and psychological dependence. Children are far more resilient, adaptable, and capable than our society typically assumes. We have segregated the young in age-graded institutions, and, as a result, children grow up with little contact with adults apart from their parents and other relatives and childcare professionals. Unlike children in the past, young people today have fewer socially valued ways to contribute to their family’s well-being or to participate in community life. By looking back over four centuries of American childhood we can perhaps recover old ways and discover new ways to reconnect children to a broader range of adult mentors and to expand their opportunities to participate in activities that they and society find truly meaningful.”

–Steven Mintz, Huck’s Raft: A History of American Childhood