List-eating functions in Lisp

“Since the Lisp philosophy strongly emphasizes the use of lists to store and manipulate information, it will come as no surprise that the design of Common Lisp favors behaviors that make it easy to slice and dice such lists. The most profound design decision made in Common Lisp, with regard to lists, is that it automatically treats an empty list as a false value when evaluating a condition […]  when we pass the empty list () into an if form, it evaluates as a false value, whereas a list that contains an item evaluates as true.

Because we can easily detect an empty list, we can process lists using recursion. With this technique, we can take items from the front of a list and send the rest of the list back to the same function until the list is empty. (It’s a good thing that detecting empty lists is so easy, because so many functions in Lisp end up being list-eaters.)

Let’s look at a common list-eating function, which calculates the length of a list:

> (defun my-length (list)
     (if list
         (1+ (my-length (cdr list)))

> (my-length '(list with four symbols))

This function is written in classic Lisp style. It calls itself recursively as it chomps items off the front of the list. Calling yourself in this way is not only allowed in Lisp, but is often strongly encouraged. Lists in Lisp are recursive (conses of conses of conses . . .), so the act of consuming a list maps naturally onto functions that are recursive.”

— Conrad Barski, The Land of Lisp: Learn to Program in Lisp, One Game at a Time (No Starch Press, 2011)

Common Lisp / Scheme macros

Here is a collection of snippets I’ve found helpful for understanding Common Lisp and Scheme macros …


“Macros are the single greatest advantage that Lisp has as a programming language and the single greatest advantage of any programming language. With them you can do things that you simply cannot do in other languages. Because macros can be used to transform lisp into other programming languages and back, programmers who gain experience with them discover that all other languages are just skins on top of lisp. This is the big deal. Lisp is special because programming with it is actually programing at a higher level. Where most languages invent and enforce syntactic and semantic rules, lisp is general and malleable. With lisp, you make the rules. […]

[…] macros are still not used to the fullest possible extent by most lisp programmers and are not used at all by all other programmers. This has always been a conundrum for advanced lispers. Since macros are so great, why doesn’t everybody use them all the time? While it’s true that the smartest, most determined programmers always end up at lisp macros, few start their programming careers there. Understanding why macros are so great requires understanding what lisp has that other languages don’t. It requires an understanding of other, less powerful languages. Sadly, most programmers lose the will to learn after they have mastered a few other languages and never make it close to understanding what a macro is or how to take advantage of one. But the top percentile of programmers in any language are always forced to learn some sort of way to write programs that write programs: macros. Because it is the best language for writing macros, the smartest and most determined and most curious programmers always end up at lisp.

Although the top-percentile of programmers is necessarily a small number, as the overall programming population grows so does the number of top-percentile programmers. The programming world sees few examples of the power of macros and understands far fewer, but this is changing. Because of the productivity multiplication that can be achieved through macros, the age of the macro is coming, whether the world is ready or not. […] Be prepared.

The conventional wisdom surrounding macros is to use them only when necessary because they can be difficult to understand, contain extremely subtle bugs, and limit you in possibly surprising ways if you think of everything as functions. These aren’t defects in the lisp macro system itself but instead are traits of macro programming in general. As with any technology, the more powerful the tool, the more ways there are to misuse it. And, as far as programming constructs go, lisp macros are the most powerful tool.

An interesting parallel to learning macros in lisp is that of learning pointers in the C programming language. Most beginning C programmers are able to quickly pick up most of the language. Functions, types, variables, arithmetic expressions: all have parallels in previous intellectual experiences beginners might have had, from elementary school maths to experimenting with simpler programming languages. But most novice C programmers hit a brick wall when they encounter pointers.

Pointers are second nature to experienced C programmers, most of whom consider their complete understanding necessary for the proper use of C. Because pointers are so fundamental, most experienced C programmers would not advise limits on their use for stylistic or learning purposes. Despite this, many C novices feel pointers are an unnecessary complication and avoid their use, resulting in the FORTRAN in any language symptom where valuable language features are neglected. The disease is ignorance of the language’s features, not poor programming style. Once the features are fully understood, the correct styles are obvious. An auxiliary theme […] is that in programming, style is not something to pursue directly. Style is necessary only where understanding is missing1.

Like C pointers, the macro is a feature of lisp that is often poorly understood, the wisdom on its proper use being very distributed and idealised. If when considering macros you find yourself relying on stylistic aphorisms like

Macros change the syntax of lisp code.

Macros work on the parse tree of your program.

Only use macros when a function won’t do.

you are probably missing the big picture when it comes to macro programming.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Introduction)

“When writing computer programs, certain patterns arise over and over again. For example, programs must often loop through the elements of arrays, increment or decrement the values of variables, and perform multi-way conditionals based on numeric or character values. Programming language designers typically acknowledge this fact by including special-purpose syntactic constructs that handle the most common patterns.  C, for instance, provides multiple looping constructs, multiple conditional constructs, and multiple constructs for incrementing or otherwise updating the value of a variable.

Some patterns are less common but can occur frequently in a certain class of programs or perhaps just within a single program. These patterns might not even be anticipated by a language’s designers, who in any case would typically choose not to incorporate syntactic constructs to handle such patterns in the language core. Yet, recognizing that such patterns do arise and that special-purpose syntactic constructs can make programs both simpler and easier to read, language designers sometimes include a mechanism for syntactic abstraction , such as C’s preprocessor macros or Common Lisp macros.[…]

Syntactic abstraction facilities differ in several significant ways. C’s preprocessor macros are essentially token-based, allowing the replacement of a macro call with a sequence of tokens with text from the macro call substituted for the macro’s formal parameters, if any. Lisp macros are expression-based, allowing the replacement of a single expression with another expression, computed in Lisp itself and based on the subforms of the macro call, if any.”

— R. Kent Dybvig, Syntactic Abstraction: The syntax-case expander

“A Lisp macro is like a function that takes Lisp forms or objects as input, and typically generates code to be then compiled and executed. This happens before runtime, in a phase called macroexpansion time. Macros can perform arbitrary computation during expansion, using the full Lisp language.

One use of macros is transforming input representing arbitrary source code into a version specified in terms of known definitions. In other words, macros can add new syntax to the original language (this is known as syntactic abstraction).

This enables easily embedding domain-specific languages, since specialized syntax can be added before runtime.

The chief benefit of macros is that they add power by letting the programmer express intent clearly and with less code. Particularly, one can add new features to the language that appear as if they were builtin. In addition, when used to pre-emptively compute data or initialize state, macros may aid in performance optimisation.”

— Abhishek Reddy, Features of Common Lisp


“An important principle of macro design is top-down programming. When designing a lisp macro, you want to start with the abstraction first. You want to write programs that use the macro long before you write the macro itself. Somewhat paradoxically, you need to know how to program in a language before you can write a concise definition/implementation for that language.

So the first step in serious macro construction is always to write use cases of the macro, even though there is no way you can test or use them. If the programs written in the new language are comprehensive enough, a good idea of what will be required to implement a compiler or interpreter for the language will follow.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Chapter 5: Programs that Program)





“In Common Lisp, the rule is simple. If the first symbol of a form is defined as a macro, the macroexpander is called with the unevaluated arguments of the form, and then the return value is recursively macroexpanded. Since everything’s an S-expression, you don’t need any more than this. Any macro you create will have the same syntax as normal Lisp forms.”

— Random stranger @ C2-Wiki …


“Macros can leverage a technique called implicit context. In code that is used frequently, or absolutely must be concise and lacking any surrounding book-keeping cruft, we sometimes choose to implicitly add lisp code around portions of an expression so we don’t have to write it every time we make use of an abstraction. We’ve talked before about implicit contexts and it should be clear that even when not programming macros they are a fundamental part of lisp programming: let and lambda forms have an implicit progn because they evaluate, in sequence, the forms in their body and return the last result.”

— Doug Hoyte, Let Over Lambda: 50 Years of Lisp (Chapter 5: Programs that Program)

“In essence, hygienic macro expansion implements lexical scoping with respect to the source code, whereas unhygienic expansion implements lexical scoping with respect to the code after expansion.”

Balanophora fungosa

Balanophora fungosa is a parasitic plant growing on the roots of rainforest plants. The flowering structure is shaped like a toadstool but consists of a globe covered with thousands of tiny female flowers. The globe is surrounded at its base by a much smaller number of male flowers. In flower, the plant emits an odour resembling that of mice.”

Young male inflorescences, flowers still unopened. 1600 m in montane cloud forest. Mt. Samkos (Pursat Province) in the Cardamom Mountains, Cambodia. (Photo Nov. 2010 by Jeremy Holden.)
Young male inflorescences, flowers still unopened. 1600 m in montane cloud forest. Mt. Samkos (Pursat Province) in the Cardamom Mountains, Cambodia. (Photo Nov. 2010 by Jeremy Holden.)

Architectural implications of NAT

“The architectural intent of NAT is to divide the Internet into independent address administrations […] The result of this division is to enforce a client/server architecture (vs. peer/peer) where the servers need to exist in the public address realm.

A significant factor in the success of the Internet is the flexibility derived from a few basic tenets. Foremost is the End- to-End principle, which notes that certain functions can only be performed in the endpoints, thus they are in control of the communication, and the network should be a simple datagram service that moves bits between these points. Restated, the endpoint applications are often the only place capable of correctly managing the data stream. Removing this concern from the lower layer packet-forwarding devices streamlines the forwarding process, contributing to system-wide efficiency.

Another advantage is that the network does not maintain per connection state information. This allows fast rerouting around failures through alternate paths and to better scaling of the overall network. Lack of state also removes any requirement for the network nodes to notify each other as endpoint connections are formed or dropped. Furthermore, the endpoints are not, and need not be, aware of any network components other than the destination, first hop router(s), and an optional name resolution service. Packet integrity is preserved through the network, and transport checksums and any address-dependent security functions are valid end-to-end.

NAT devices (particularly the NAPT variety) undermine most of these basic advantages of the end-to-end model, reducing overall flexibility, while often increasing operational complexity and impeding diagnostic capabilities.”

From IETF RFC 2993: “Architectural Implications of NAT”, November 2000

Jimmy Carter’s Blood-Drenched Legacy

Two Young Girls Found Alongside the Highway to Comalapa Airport Photo: John Hoagland
Two Young Girls Found Alongside the Highway to Comalapa Airport, El Salvador (August 1980) Photo: John Hoagland

“[…] the self-professed advocate for human rights demonstrated quite the penchant for bloodshed. While he didn’t initiate any aggressive invasions of foreign nations the way his predecessors and successors did in Vietnam, Grenada, Panama, Iraq, Afghanistan and many other countries, Carter proved remarkably generous at providing financial, military, diplomatic and ideological support for fascist dictatorships that tortured and killed millions of members of their domestic populations in an effort to crush popular movements for social justice. Some of the regimes he backed carried out mass slaughter that amounted to genocide.

Below are some of Carter’s most shameful and indefensible foreign policy positions that caused monumental levels of death, destruction and suffering for poor, socially disenfranchised people from Asia to Latin America to Africa.” […]

Read the full article at Counterpunch:

Proudhon: Your employer is your enemy.

“The civilised labourer who bakes a loaf that he may eat a slice of bread, who builds a palace that he may sleep in a stable, who weaves rich fabrics that he may dress in rags, who produces every thing that he may dispense with every thing, — is not free. His employer, not becoming his associate in the exchange of salaries or services which takes place between them, is his enemy.”

— Pierre Joseph Proudhon

Water: source of life and conflict in the Land of Rivers

(by Joris Leverink, via ROARMag)

Iraq: Kurmashia Marsh: February 18, 2004: A Marsh Arab poles his canoe through Kirmashiya Marsh in southern Iraq.
Iraq: Kurmashia Marsh: February 18, 2004: A Marsh Arab poles his canoe through Kirmashiya Marsh in southern Iraq.

Where oil is widely considered to be one of the main causes for the region’s instability — mainly because it drew imperialist powers to the region that eagerly supported local dictators to ensure continued and unlimited access to the precious liquid — another potential source of conflict is often overlooked. Water, the first and foremost source of life in the barren desert regions of the Middle East, which allowed for the world’s first civilizations to develop on the fertile floodplains between the Euphrates and Tigris rivers, is becoming ever more scarce, and the struggles to safeguard a fair share are growing fiercer by the day. […]

On several occasions over the past decades local development projects on the Euphrates and Tigris rivers have brought the three neighboring states Turkey, Syria and Iraq to the brink of war. When in 1990 Turkey blocked the flow of the Euphrates for nine days to fill the reservoir of the Atatürk dam, Iraq massed troops on its border and threatened to bomb the dam. Nowadays, tensions remain high as yet another Turkish mega-dam is about to be completed — the Ilisu dam on the Tigris river — which will severely reduce the water flow to Iraq and destroy thousands of years of cultural and historical heritage at home. […]

Topping the list of concerns of many local and international campaigns against the construction of the Ilisu dam is the fate of the town of Hasankeyf. The town and its surroundings are home to numerous archaeological sites — some of which remain unexplored — that date back more than 12,000 years. The ruins of an 11th century bridge mark the spot where the Silk Road once crossed the river Tigris and the thousands of human-made caves that dot the mountains bare witness to the unique culture of the region. All of this is set to disappear below the surface of the water once the inundation of the dam reservoir begins.

Immediately after the announcement of the project in 1997, a social movement emerged. Civil society groups, local professionals and international NGOs joined forces to oppose the project and raise awareness about the potential destruction of the natural environment, the cultural heritage and the displacement of up to 78,000 people from their homes in and around Hasankeyf.

A successful international campaign temporarily halted the project in 2009, when a number of European financiers withdrew their support after it was exposed that Turkey failed to meet the international standards of dam-building set by the World Bank to protect the environment, affected people, riparian states and cultural heritage. However, after Turkey turned to its national banks to provide the necessary funding, the project is now back on track and is set for completion this year. […]

The Ilisu dam is part of the giant Southeast Anatolia Regional Development Project (GAP, after its Turkish acronym) which was launched in 1977 and aims to built a total of 22 dams and 19 hydroelectric power plants by 2015, covering nine provinces in southeastern Turkey. The GAP project is presented by the government as bringing development to the traditionally impoverished and underdeveloped regions where poor living standards have caused the local Kurds to rise up against the central state for many decades.

For years, the Turkish central government, led by the former prime minister and current president Erdogan, has claimed that there is no such thing as a “Kurdish problem”, denying the fact that the country’s Kurdish population has been discriminated against on the basis of its ethnic background, and arguing that the Kurds’ hardships stem from the underdevelopment of their traditional homelands in southeastern Turkey. […]

The finished GAP project will reduce water flows to Syria by 40 percent, and to Iraq by a shocking 80 percent. This, in combination with the severe droughts that have hit the region over the past few years, the ongoing conflict between the Iraqi state and it allies and the militants of the so-called Islamic State, and the millions of (internally) displaced people in the region, has the potential to unleash an environmental and humanitarian catastrophe that could cause a serious food security problem, destabilizing the region for years to come. […]

From the Marsh Arabs in southern Iraq to the Kurds in Turkey, the struggle for equal access to the Earth’s resources is connected across ethnic, religious and national boundaries. As such, it provides a unique opportunity to raise awareness about the interdependence of the region’s communities, forging bonds that transcend the interests of central governments and international powers.


Read the full article at ROARMag:


We Wake The Day

We wake; we wake the day,
the light rising in us like sun–
our breath a prayer brushing
against the feathers in our hands.
We stumble out into streets;
patterns of wire invented by strangers
are strung between eye and sky,
and we dance in two worlds,
inevitable as seasons in one,
exotic curiosities in the other
which rushes headlong down highways,
watches us from car windows, explains
us to its children in words
that no one could ever make
sense of. The image obscures
the vision, and we wonder
whether anyone will ever hear
our own names for the things
we do. Light dances in the body,
surrounds all living things–
even the stones sing
although their songs are infinitely
slower than the ones we learn
from trees. No human voice lasts
long enough to make such music sound.
Earth breath eddies between factories
and office buildings, caresses the surface
of our skin; we go to jobs, the boss
always watching the clock to see
that we’re on time. He tries to shut
out magic and hopes we’ll make
mistakes or disappear. We work
fast and steady and remember
each breath alters the composition
of the air. Change moves relentless,
the pattern unfolding despite their planning–
we’re always there–singing round dance
songs, remembering what supports
our life–impossible to ignore.

-by Gail Tremblay, in Reinventing the Enemy’s Language