Chapter 33: Programmable Completion

Introduction

Programmable completion, or autocompletion, is a powerful feature that enhances command-line efficiency by suggesting or completing commands, arguments, filenames, and other elements as you type. It simplifies interactions with the terminal, helping users enter commands quickly and accurately while minimizing manual effort.

This feature offers several key benefits. It saves time by reducing the need to type long commands or filenames in full; a few characters followed by pressing the Tab key allow Bash to complete the input. Autocompletion also helps prevent errors by suggesting valid options based on the current context, ensuring accuracy in command entry. Furthermore, it enhances discoverability by displaying a list of available commands, options, or files, making it easier to explore and utilize system resources effectively.

Autocompletion is particularly useful for working with command parameters, as it can display available flags, options, and arguments for specific commands. This aids users in constructing precise and meaningful commands without relying on memory alone. It’s also invaluable when developing shell scripts, suggesting valid command names, system variables, or file paths to ensure accuracy and streamline the scripting process. For complex commands involving multiple arguments or filenames, autocompletion reduces both the effort required and the likelihood of mistakes.

In essence, Bash autocompletion boosts productivity and precision on the command line. By providing an intuitive and efficient way to interact with the shell, it simplifies command entry, aids discovery, and makes working in the terminal more seamless.

In this chapter, we will explore the environment variables and built-in commands that enable programmable completion for various commands of our choosing. To apply what we learn, we will create a dummy script designed to display the arguments provided to it, and then use that script as the basis for building a programmable completion script. Let’s begin with an overview of the relevant environment variables.

Programmable completion environment variables

The following sections provide detailed descriptions of the key variables used in scripts to enable programmable completion in Bash.

The “COMP_CWORD” variable

The “COMP_CWORD” variable serves as an index within the “COMP_WORDS” array, identifying the word currently being autocompleted. This variable functions as an input variable and is accessible only within shell functions invoked by Bash’s programmable completion framework.

The “COMP_KEY” variable

The “COMP_KEY” variable holds the ASCII code [1]of the key pressed to trigger the current completion function. For most cases, this value will be 9, which represents the Tab key. Like other input variables, it is limited to use within shell functions called by Bash’s completion mechanisms.

The “COMP_LINE” variable

The “COMP_LINE” variable contains the entire command line that is being autocompleted. It provides the full context for the autocomplete process and is also an input variable available exclusively within Bash’s completion-related shell functions.

The “COMP_POINT” variable

The “COMP_POINT” variable represents the cursor’s position within the command line. If the cursor is at the end of the command, its value will equal the length of “COMP_LINE”. This variable is crucial for determining where completion should occur and, like others, is an input variable used only in Bash’s completion facilities.

The “COMP_TYPE” variable

The “COMP_TYPE” variable indicates the type of completion that triggered the function call, represented by an integer ASCII code. The possible values include:

• 9 (Tab): Normal completion
• 33 (“!”): Lists alternatives for partially completed words
• 37 (“%”): Indicates menu completion
• 63 (“?”): Lists completions after multiple Tab presses
• 64 (“@”): Lists completions for unmodified words This variable is an input value, confined to completion-specific shell functions.

The “COMP_WORDBREAKS” variable

The “COMP_WORDBREAKS” variable defines the characters considered as word separators by the “readline” library during completion. Modifying this variable removes its special properties, so it is recommended not to alter it.

The “COMPREPLY” variable

The “COMPREPLY” variable is an array used to store suggestions for autocompletion. These suggestions are generated within a shell function and read by Bash to present possible completions to the user. Unlike the input variables above, “COMPREPLY” serves as an output variable.

The “COMP_WORDS” variable

The “COMP_WORDS” variable is an array containing the individual words of the current command line. It acts as an input variable, providing the components of the command for processing. Like other input variables, it is only accessible within Bash’s completion-related functions.

With an understanding of these variables, the next step is to explore the built-in commands that work in conjunction with them to create and manage programmable completion functionality effectively.

Programmable completion built-in commands

In the following sections, we will explore the built-in commands that Bash provides to create programmable completion scripts. These commands serve as essential tools for enhancing command-line efficiency by offering dynamic and context-sensitive suggestions.

The “compgen” Command

The “compgen” command is a powerful built-in tool in Bash designed to facilitate command-line completion. Its primary function is to generate a list of possible completions based on specific criteria, such as command names, file names, or environment variables. When a user begins typing a command or argument and presses the Tab key, Bash relies on “compgen” to propose relevant suggestions, simplifying navigation and command entry.

“compgen” can be utilized both interactively on the command line and within scripts, making it versatile for generating completions. It supports a wide range of completion types, including commands, options, files, directories, usernames, and more. By leveraging “compgen”, users can quickly view valid options or arguments for a command, reducing the need to recall or type them manually and minimizing the likelihood of errors.

In essence, “compgen” enhances the user experience in Bash by streamlining command-line operations, providing immediate access to available resources, and reducing typing errors. Its flexibility allows it to be adapted for various scenarios, improving both usability and productivity.

The “compgen” command offers numerous options and flags that can be used to tailor suggestions for programmable completion. These flags and options collectively form what is known as a “compspec” (completion specification). A compspec defines the criteria that guide “compgen” in determining the suggestions it generates. In this section and subsequent sections, you will learn about various flags and options that can be passed to “compgen” to produce the desired completions for commands.

The following table outlines the initial and most straightforward flags of the compgen command, along with descriptions of their associated behaviors:

While these flags provide a solid foundation, “compgen” also includes more advanced options for generating and modifying suggestions. These capabilities make it an indispensable tool for developing sophisticated programmable completion scripts. Typically, “compgen” is invoked within shell functions specifically designed to generate the possible completions, ensuring precise and context-appropriate suggestions.

Actions (the “-A” flag)

The “-A” flag is part of the compspec. In a nutshell, this flag admits an action as an argument. Depending on the value passed, you will have one or another suggestion.

The following table provides an overview of the different values you can pass as action to the “-A” flag, with a description of the behavior.

Function (the “-F” flag)

Using the “-F” flag, you can specify the name of a function to generate a custom list of suggestions for your command.

Within this function, you’ll have access to various environment variables described in the section on programmable completion environment variables.

The sole requirement for the function is to populate the “COMPREPLY” array with the desired suggestions. As we’ve learned earlier, Bash uses this array to display the suggestions for your command.

Globbing (the “-G” flag)

The “-G” flag enables you to provide an expression as its argument, which will be expanded to match file names. This expression supports various wildcards, allowing for more flexible and dynamic pattern matching.

Word list (the “-W” flag)

The “-W” flag allows you to specify a string containing a list of items that will serve as suggestions for your command when using programmable completion. This means you can directly define a set of predefined options that Bash will present as autocompletion suggestions. By including this flag, you can ensure that users of your command are guided toward selecting from a specific range of values, making the interaction more intuitive and efficient. This approach is particularly useful when the list of potential options is fixed or easily determined in advance.

Filter out (the “-X” flag)

The “-X” flag provides a way to refine the suggestions generated for your command by excluding specific options that match a pattern you specify as its argument. This pattern serves as a filter, ensuring that any suggestions aligning with it are removed from the list of completions presented to the user.

However, the “-X” flag does not generate suggestions on its own. Instead, it works in conjunction with other flags, serving as a complementary tool to exclude unwanted matches from an existing set of completions. By using this flag effectively, you can create a more tailored and user-friendly set of suggestions, ensuring that irrelevant or redundant options are omitted.

Suffix (the “-S” flag)

The “-S“[2] flag enables you to append a specified suffix to every suggestion generated during programmable completion. By providing the suffix as an argument to this flag, you can ensure that each suggestion includes the desired text appended to the end, enhancing the clarity or functionality of the completion results.

It’s important to note that the “-S” flag does not operate independently. Instead, it works in tandem with other flags that generate the actual suggestions. Its role is purely to modify these existing suggestions by adding the specified suffix. This feature can be particularly useful in scenarios where a consistent format or additional context is needed for each suggestion.

Options (the “-o” flag)

The “-o” flag introduces specific behaviors that take effect once the completion specification (compspec) has been evaluated. This flag allows you to fine-tune how the completion system behaves, adding flexibility and control to the overall completion process.

To help you better understand its functionality, the table below outlines the various option values you can pass to the “-o” flag. Each value is accompanied by a description detailing the corresponding behavior it activates, making it easier to determine the most appropriate choice for your needs.

The “complete” Command

The “complete” command serves as the primary tool for implementing programmable completion in Bash. It can be used on its own or in conjunction with the “compgen” command, which we explored in the previous section.

The “complete” command offers the same options as “compgen”, with the addition of two unique flags: “-p” and “-r”.

Now that you understand the roles of both “compgen” and “complete”, it’s time to delve into how these commands can be effectively combined. This topic will be the focus of the next section.

How to use “compgen” and “complete” together?

So far, we have explored the purpose of the compgen and complete commands, understanding their roles in programmable completion. However, to enable programmable completion effectively, these commands must be used together.

In this section, we will learn how to combine these two commands to implement programmable completion. But before diving into the details, let’s start by creating a very simple script as the basis for our example. This lightweight script will be the target for which we’ll provide programmable completion.

Here’s the script:

 1 #!/usr/bin/env bash
 2 #Script: programmable-completion-0001.sh
 3 echo "COMMAND: $0 $@"

As you can see, the script’s sole purpose is to print the string “COMMAND: ” followed by the script’s name and any arguments passed to it. It’s minimal by design, serving as a straightforward example for our completion logic.

Now, how do we add programmable completion to this script? As mentioned earlier, we need to leverage the built-in “compgen” and “complete” commands. To achieve this, we’ll create a separate script that contains the logic for handling programmable completion.

Here’s the completion script:

 1 #!/usr/bin/env bash
 2 #Script: programmable-completion-0001-completion.bash
 3 _mucommand_completions() {
 4 		"[[LOGIC_TAKING_CARE_OF_PROGRAMMABLE_COMPLETION]]"
 5 }
 6 complete -F _mycommand_completions programmable-completion-0001

In this script, pay close attention to line 6, where the “complete” command is used. This command employs the “-F” flag to associate a specific function—named “_mycommand_completions” in our example—with the command “programmable-completion-0001”. By sourcing this file, Bash is informed that whenever “programmable-completion-0001” is invoked, it must call the “_mycommand_completions” function to generate a list of suggestions.

After the “_mycommand_completions” function executes, Bash retrieves the suggestions stored in the “COMPREPLY” array and presents them to the user.

Within the “_mycommand_completions” function, you’ll define the logic for generating suggestions. This typically involves using the “compgen” command to dynamically build the list of possible completions[3].

In the subsections that follow, we’ll explore a series of examples to help you become comfortable with programmable completion. Each example will introduce progressively more advanced concepts, ensuring a clear learning path. I hope you find this journey both engaging and enjoyable!

Adding static values to the “COMPREPLY” array variable

In our first example, we will start by adding static values to the “COMPREPLY” array.

 1 #!/usr/bin/env bash
 2 #Script: programmable-completion-0002-completion.bash
 3 _mycommand_completions(){
 4     COMPREPLY=("opt1" "opt2" "opt3")
 5 }
 6 complete -F _mycommand_completions programmable-completion-0001

In this version of the script, we are manually defining three fixed suggestions: “opt1”, “opt2”, and “opt3”. Once you source[4] this script in your current terminal session, you can test it by typing “programmable-completion-0001“[5] and pressing the Tab key.

When you press Tab for the first time, Bash will suggest “opt”, as all available completions share this common prefix. Pressing Tab twice more will reveal the full list of suggestions:

$ programmable-completion-0001 opt
opt1  opt2  opt3

At this point, you can choose the option you want. For example, if you select “opt2” and press Enter, the command will execute with the following output:

$ programmable-completion-0001 opt2
COMMAND: programmable-completion-0001 opt2
$

With this approach, Bash will always provide the static values stored in the “COMPREPLY” array as suggestions.

In the next section, we’ll explore how to use the “compgen” command along with its various flags to dynamically generate completions.

Adding static values with “compgen”

In this section, we will explore how to use the “compgen” command to generate static suggestions for programmable completion.

For this example, our completion script will provide suggestions for file names (using the “-f” flag), group names (“-g” flag), and user names (“-u” flag).

 1 #!/usr/bin/env bash
 2 #Script: programmable-completion-0003-completion.bash
 3 _mycommand_completions(){
 4 		COMPREPLY=($(compgen -f -g -u))
 5 }
 6 complete -F _mycommand_completions programmable-completion-0001

After saving the script, source it in your terminal as you did previously, and then test it by typing the command followed by the Tab key. You should see output similar to the following:

$ programmable-completion-0001 <Tab><Tab>
adm              gnome-initial-setup             sgx
_apt             hplip                           shadow
audio            input                           speech-dispatcher
...

Keep in mind that the actual output will vary depending on your system’s files, users, and groups.

While this script improves upon the previous example by generating dynamic suggestions, it still has a limitation: it suggests the same options repeatedly, without considering what the user has already typed.

In the next section, we’ll address this issue and refine our completion logic to make it more intelligent and context-aware.

Dynamic completion based on what has been typed so far

Building upon the previous section, we will once again use the “compgen” command—this time incorporating the user’s input so that the suggestions provided are more relevant to what has been typed so far.

To achieve this, we need to utilize the “COMP_WORDS” array variable. As a reminder, this array holds the individual words in the current command line, where each word is indexed sequentially.

Since we want to provide autocompletion for the first argument after the command itself, we need to reference index “1” of the “COMP_WORDS” array. This is because index “0” contains the command name (“programmable-completion-0001”), while index “1” represents the first argument the user is typing.

Here’s how our improved completion script looks:

 1 #!/usr/bin/env bash
 2 #Script: programmable-completion-0004-completion.bash
 3 _mycommand_completions(){
 4 		COMPREPLY=($(compgen -f -g -u ${COMP_WORDS[1]}))
 5 }
 6 complete -F _mycommand_completions programmable-completion-0001

With this approach, Bash will only suggest options that begin with the characters you have already typed. For example, after sourcing the script, if you start typing the command and enter the letter “v”, then press Tab, Bash will filter the suggestions to display only those that start with “v”:

$ programmable-completion-0001 v<Tab><Tab>
vboxusers  video      voice

Now, let’s say you select “video” from the suggestions. If you press Tab again, you might notice an unusual behavior—Bash will repeatedly suggest “video” without offering any new completions.

This happens because our script is always referencing the same position in the “COMP_WORDS” array, which now permanently holds the value “video”. As a result, it keeps generating the same suggestion repeatedly.

To fix this issue, we need to dynamically determine which position in the command line we are providing suggestions for.

In the next section, we’ll explore how to refine our script to handle this scenario more effectively.

Dynamic completion with several arguments and sub-arguments

In this section, we will create a programmable completion script for a command that takes three main arguments, each with two corresponding sub-arguments.

To implement this feature correctly, we must clearly identify which “COMP_CWORD” values need to be checked. The diagram below illustrates this concept:

From the diagram, we can see that:

• To provide argument suggestions, we check the “COMP_CWORD” value.
• To suggest sub-arguments, we check both “COMP_CWORD” and the selected argument.

Step 1: Suggesting Arguments

Let’s begin by implementing argument suggestions using the “-W” option of the “compgen” built-in command.

 1 #!/usr/bin/env bash
 2 # Script: mycommand-completion.bash
 3 _mycommand_completions() {
 4    if [[ "$COMP_CWORD" == 1 ]]; then
 5       COMPREPLY=($(compgen -W "user video image" ${COMP_WORDS[1]}))
 6       return
 7    fi
 8 }
 9 complete -F _mycommand_completions mycommand

Now, source the script and test it:

$ ./mycommand <Tab><Tab>
image  user   video

At this stage, selecting an argument and pressing Tab again should not produce any further suggestions:

$ ./mycommand image <Tab><Tab>

Great job! We have successfully provided argument completion. Now, let’s move on to sub-arguments.

Step 2: Suggesting Sub-Arguments

As mentioned earlier, to provide sub-argument suggestions, we need to:

1. Ensure “COMP_CWORD” is set to “2”.
2. Identify which argument is currently selected.

To achieve this, we store the argument in a variable and use conditional checks to provide appropriate sub-argument suggestions.

 1 #!/usr/bin/env bash
 2 # Script: mycommand-completion.bash
 3 _mycommand_completions() {
 4    if [[ "$COMP_CWORD" == 1 ]]; then
 5       COMPREPLY=($(compgen -W "user video image" ${COMP_WORDS[1]}))
 6       return
 7    fi
 8    local argument=${COMP_WORDS[ (( COMP_CWORD - 1 )) ]}
 9    if [[ "$COMP_CWORD" == 2 ]]; then
10       case "$argument" in
11          image) COMPREPLY=($(compgen -W "resize rotate" ${COMP_WORDS[2]})) ;;
12          user)  COMPREPLY=($(compgen -W "add remove" ${COMP_WORDS[2]})) ;;
13          video) COMPREPLY=($(compgen -W "compress record" ${COMP_WORDS[2]})) ;;
14       esac
15       return
16    fi
17 }
18 complete -F _mycommand_completions mycommand

Testing Sub-Argument Completion

Source the script again and test the completion process:

$ ./mycommand <Tab><Tab>
image  user   video

Now select “image” and press Tab again:

$ ./mycommand image <Tab><Tab>

At first, you’ll only see the letter “r”, as both available sub-arguments (“resize” and “rotate”) start with this letter.

$ ./mycommand image r

Pressing Tab again will display the full options:

$ ./mycommand image r<Tab><Tab>
resize  rotate

Wrapping Up

If everything is working as expected, congratulations! You’ve successfully implemented programmable completion for both arguments and sub-arguments.

If you encounter issues, don’t worry—double-check your script and try again. Debugging is a natural part of the learning process.

While we could explore more advanced examples, you now have a solid understanding of how to create programmable completion scripts.

In the following section, we’ll learn where to place these scripts so that Bash loads them automatically when you start a new terminal session.

Summary

References

1. https://www.ascii-code.com/↩

2. There is another flag that is supposed to have a similar behavior to attach prefixes. The flag is “-P”. But every time I tried to use it does not behave consistently as the “-S” flag. Funny thing is that if you use the “-P” flag from Zsh it provides the expected behavior.↩

3. For the record, using the "compgen" command is not mandatory. However, it is an incredibly powerful tool designed to simplify your work and enhance efficiency.↩

4. You can use the command “source programmable-completion-0002-completion.bash” to make the changes effective in the current terminal.↩

5. “./programmable-completion-0001” because it is in the current folder. If have the script in a folder that is the “PATH” variable you can remove the “./” and just use “programmable-completion-0001”.↩