MacroExpander: Allow registering local expansions

This allows a MacroExpander also to describe an expansion it can
do which was restricted to the global VariableManager only.

The global is now just a thin (unneeded) wrapper about new "standard"
functionality.

Change-Id: Ida7ca70cf3d319eae4220ea8d12f3dd1c0d4042c
Reviewed-by: Tobias Hunger <tobias.hunger@theqtcompany.com>

src/libs/utils/macroexpander.cpp

@@ -30,15 +30,294 @@
 
 #include "macroexpander.h"
 
+#include <QCoreApplication>
+#include <QDebug>
+#include <QFileInfo>
+#include <QMap>
+
 namespace Utils {
+namespace Internal {
+
+const char kFilePathPostfix[] = ":FilePath";
+const char kPathPostfix[] = ":Path";
+const char kFileNamePostfix[] = ":FileName";
+const char kFileBaseNamePostfix[] = ":FileBaseName";
+
+class MacroExpanderPrivate
+{
+public:
+    QHash<QByteArray, MacroExpander::StringFunction> m_map;
+    QHash<QByteArray, MacroExpander::PrefixFunction> m_prefixMap;
+    QMap<QByteArray, QString> m_descriptions;
+};
+
+} // Internal
+
+using namespace Internal;
+
+/*!
+    \class Utils::MacroExpander
+    \brief The MacroExpander class manages \QC wide variables, that a user
+    can enter into many string settings. The variables are replaced by an actual value when the string
+    is used, similar to how environment variables are expanded by a shell.
+
+    \section1 Variables
+
+    Variable names can be basically any string without dollar sign and braces,
+    though it is recommended to only use 7-bit ASCII without special characters and whitespace.
+
+    If there are several variables that contain different aspects of the same object,
+    it is convention to give them the same prefix, followed by a colon and a postfix
+    that describes the aspect.
+    Examples of this are \c{CurrentDocument:FilePath} and \c{CurrentDocument:Selection}.
+
+    When the variable manager is requested to replace variables in a string, it looks for
+    variable names enclosed in %{ and }, like %{CurrentDocument:FilePath}.
+
+    Environment variables are accessible using the %{Env:...} notation.
+    For example, to access the SHELL environment variable, use %{Env:SHELL}.
+
+    \note The names of the variables are stored as QByteArray. They are typically
+    7-bit-clean. In cases where this is not possible, UTF-8 encoding is
+    assumed.
+
+    \section1 Providing Variable Values
+
+    Plugins can register variables together with a description through registerVariable().
+    A typical setup is to register variables in the Plugin::initialize() function.
+
+    \code
+    bool MyPlugin::initialize(const QStringList &arguments, QString *errorString)
+    {
+        [...]
+        MacroExpander::registerVariable(
+            "MyVariable",
+            tr("The current value of whatever I want."));
+            []() -> QString {
+                QString value;
+                // do whatever is necessary to retrieve the value
+                [...]
+                return value;
+            }
+        );
+        [...]
+    }
+    \endcode
 
-MacroExpander::MacroExpander(const MacroExpander::Resolver &resolver)
-    : m_resolver(resolver)
-{}
 
+    For variables that refer to a file, you should use the convenience function
+    MacroExpander::registerFileVariables().
+    The functions take a variable prefix, like \c MyFileVariable,
+    and automatically handle standardized postfixes like \c{:FilePath},
+    \c{:Path} and \c{:FileBaseName}, resulting in the combined variables, such as
+    \c{MyFileVariable:FilePath}.
+
+    \section1 Providing and Expanding Parametrized Strings
+
+    Though it is possible to just ask the variable manager for the value of some variable in your
+    code, the preferred use case is to give the user the possibility to parametrize strings, for
+    example for settings.
+
+    (If you ever think about doing the former, think twice. It is much more efficient
+    to just ask the plugin that provides the variable value directly, without going through
+    string conversions, and through the variable manager which will do a large scale poll. To be
+    more concrete, using the example from the Providing Variable Values section: instead of
+    calling \c{MacroExpander::value("MyVariable")}, it is much more efficient to just ask directly
+    with \c{MyPlugin::variableValue()}.)
+
+    \section2 User Interface
+
+    If the string that you want to parametrize is settable by the user, through a QLineEdit or
+    QTextEdit derived class, you should add a variable chooser to your UI, which allows adding
+    variables to the string by browsing through a list. See Core::VariableChooser for more
+    details.
+
+    \section2 Expanding Strings
+
+    Expanding variable values in strings is done by "macro expanders".
+    Utils::AbstractMacroExpander is the base class for these, and the variable manager
+    provides an implementation that expands \QC variables through
+    MacroExpander::macroExpander().
+
+    There are several different ways to expand a string, covering the different use cases,
+    listed here sorted by relevance:
+    \list
+    \li Using MacroExpander::expandedString(). This is the most comfortable way to get a string
+        with variable values expanded, but also the least flexible one. If this is sufficient for
+        you, use it.
+    \li Using the Utils::expandMacros() functions. These take a string and a macro expander (for which
+        you would use the one provided by the variable manager). Mostly the same as
+        MacroExpander::expandedString(), but also has a variant that does the replacement inline
+        instead of returning a new string.
+    \li Using Utils::QtcProcess::expandMacros(). This expands the string while conforming to the
+        quoting rules of the platform it is run on. Use this function with the variable manager's
+        macro expander if your string will be passed as a command line parameter string to an
+        external command.
+    \li Writing your own macro expander that nests the variable manager's macro expander. And then
+        doing one of the above. This allows you to expand additional "local" variables/macros,
+        that do not come from the variable manager.
+    \endlist
+
+*/
+
+/*!
+ * \internal
+ */
+MacroExpander::MacroExpander()
+{
+    d = new MacroExpanderPrivate;
+}
+
+/*!
+ * \internal
+ */
+MacroExpander::~MacroExpander()
+{
+    delete d;
+}
+
+/*!
+ * \internal
+ */
 bool MacroExpander::resolveMacro(const QString &name, QString *ret)
 {
-    return m_resolver(name, ret);
+    bool found;
+    *ret = value(name.toUtf8(), &found);
+    return found;
+}
+
+/*!
+ * Returns the value of the given \a variable. If \a found is given, it is
+ * set to true if the variable has a value at all, false if not.
+ */
+QString MacroExpander::value(const QByteArray &variable, bool *found)
+{
+    StringFunction sf = d->m_map.value(variable);
+    if (sf) {
+        if (found)
+            *found = true;
+        return sf();
+    }
+
+    for (auto it = d->m_prefixMap.constBegin(); it != d->m_prefixMap.constEnd(); ++it) {
+        if (variable.startsWith(it.key())) {
+            PrefixFunction pf = it.value();
+            if (found)
+                *found = true;
+            return pf(QString::fromUtf8(variable.mid(it.key().count())));
+        }
+    }
+    if (found)
+        *found = false;
+
+    return QString();
+}
+
+/*!
+ * Returns \a stringWithVariables with all variables replaced by their values.
+ * See the MacroExpander overview documentation for other ways to expand variables.
+ *
+ * \sa MacroExpander
+ * \sa macroExpander()
+ */
+QString MacroExpander::expandedString(const QString &stringWithVariables)
+{
+    return Utils::expandMacros(stringWithVariables, this);
+}
+
+/*!
+ * Makes the given string-valued \a prefix known to the variable manager,
+ * together with a localized \a description.
+ *
+ * The \a value PrefixFunction will be called and gets the full variable name
+ * with the prefix stripped as input.
+ *
+ * \sa registerVariables(), registerIntVariable(), registerFileVariables()
+ */
+void MacroExpander::registerPrefix(const QByteArray &prefix, const QString &description,
+                                   const MacroExpander::PrefixFunction &value)
+{
+    QByteArray tmp = prefix;
+    if (!tmp.endsWith(':'))
+        tmp.append(':');
+    d->m_descriptions.insert(tmp + "<value>", description);
+    d->m_prefixMap.insert(tmp, value);
+}
+
+/*!
+ * Makes the given string-valued \a variable known to the variable manager,
+ * together with a localized \a description.
+ *
+ * \sa registerFileVariables(), registerIntVariable(), registerPrefix()
+ */
+void MacroExpander::registerVariable(const QByteArray &variable,
+    const QString &description, const StringFunction &value)
+{
+    d->m_descriptions.insert(variable, description);
+    d->m_map.insert(variable, value);
+}
+
+/*!
+ * Makes the given integral-valued \a variable known to the variable manager,
+ * together with a localized \a description.
+ *
+ * \sa registerVariable(), registerFileVariables(), registerPrefix()
+ */
+void MacroExpander::registerIntVariable(const QByteArray &variable,
+    const QString &description, const MacroExpander::IntFunction &value)
+{
+    const MacroExpander::IntFunction valuecopy = value; // do not capture a reference in a lambda
+    registerVariable(variable, description,
+        [valuecopy]() { return QString::number(valuecopy ? valuecopy() : 0); });
+}
+
+/*!
+ * Convenience function to register several variables with the same \a prefix, that have a file
+ * as a value. Takes the prefix and registers variables like \c{prefix:FilePath} and
+ * \c{prefix:Path}, with descriptions that start with the given \a heading.
+ * For example \c{registerFileVariables("CurrentDocument", tr("Current Document"))} registers
+ * variables such as \c{CurrentDocument:FilePath} with description
+ * "Current Document: Full path including file name."
+ *
+ * \sa registerVariable(), registerIntVariable(), registerPrefix()
+ */
+void MacroExpander::registerFileVariables(const QByteArray &prefix,
+    const QString &heading, const StringFunction &base)
+{
+    registerVariable(prefix + kFilePathPostfix,
+         QCoreApplication::translate("Utils::MacroExpander", "%1: Full path including file name.").arg(heading),
+         [base]() -> QString { QString tmp = base(); return tmp.isEmpty() ? QString() : QFileInfo(tmp).filePath(); });
+
+    registerVariable(prefix + kPathPostfix,
+         QCoreApplication::translate("Utils::MacroExpander", "%1: Full path excluding file name.").arg(heading),
+         [base]() -> QString { QString tmp = base(); return tmp.isEmpty() ? QString() : QFileInfo(tmp).path(); });
+
+    registerVariable(prefix + kFileNamePostfix,
+         QCoreApplication::translate("Utils::MacroExpander", "%1: File name without path.").arg(heading),
+         [base]() -> QString { QString tmp = base(); return tmp.isEmpty() ? QString() : QFileInfo(tmp).fileName(); });
+
+    registerVariable(prefix + kFileBaseNamePostfix,
+         QCoreApplication::translate("Utils::MacroExpander", "%1: File base name without path and suffix.").arg(heading),
+         [base]() -> QString { QString tmp = base(); return tmp.isEmpty() ? QString() : QFileInfo(tmp).baseName(); });
+}
+
+/*!
+ * Returns all registered variable names.
+ *
+ * \sa registerVariable()
+ * \sa registerFileVariables()
+ */
+QList<QByteArray> MacroExpander::variables()
+{
+    return d->m_descriptions.keys();
+}
+
+/*!
+ * Returns the description that was registered for the \a variable.
+ */
+QString MacroExpander::variableDescription(const QByteArray &variable)
+{
+    return d->m_descriptions.value(variable);
 }
 
 } // namespace Utils

src/libs/utils/macroexpander.h

@@ -37,17 +37,44 @@
 
 namespace Utils {
 
+namespace Internal { class MacroExpanderPrivate; }
+
 class QTCREATOR_UTILS_EXPORT MacroExpander : public AbstractMacroExpander
 {
 public:
-    typedef std::function<bool(const QString &name, QString *ret)> Resolver;
-
-    explicit MacroExpander(const Resolver &resolver);
+    explicit MacroExpander();
+    ~MacroExpander();
 
     bool resolveMacro(const QString &name, QString *ret);
 
+    QString value(const QByteArray &variable, bool *found = 0);
+
+    QString expandedString(const QString &stringWithVariables);
+
+    typedef std::function<QString(QString)> PrefixFunction;
+    typedef std::function<QString()> StringFunction;
+    typedef std::function<int()> IntFunction;
+
+    void registerPrefix(const QByteArray &prefix,
+        const QString &description, const PrefixFunction &value);
+
+    void registerVariable(const QByteArray &variable,
+        const QString &description, const StringFunction &value);
+
+    void registerIntVariable(const QByteArray &variable,
+        const QString &description, const IntFunction &value);
+
+    void registerFileVariables(const QByteArray &prefix,
+        const QString &heading, const StringFunction &value);
+
+    QList<QByteArray> variables();
+    QString variableDescription(const QByteArray &variable);
+
 private:
-    Resolver m_resolver;
+    MacroExpander(const MacroExpander &) Q_DECL_EQ_DELETE;
+    void operator=(const MacroExpander &) Q_DECL_EQ_DELETE;
+
+    Internal::MacroExpanderPrivate *d;
 };
 
 } // namespace Utils

src/plugins/coreplugin/dialogs/externaltoolconfig.cpp

@@ -595,5 +595,5 @@ void ExternalToolConfig::addCategory()
 void ExternalToolConfig::updateEffectiveArguments()
 {
     ui->arguments->setToolTip(Utils::QtcProcess::expandMacros(ui->arguments->text(),
-            Core::VariableManager::macroExpander()));
+            globalMacroExpander()));
 }

src/plugins/coreplugin/editormanager/editormanager.cpp

@@ -483,20 +483,20 @@ void EditorManagerPrivate::init()
     d->m_openEditorsFactory = new OpenEditorsViewFactory();
     ExtensionSystem::PluginManager::addObject(d->m_openEditorsFactory);
 
-    VariableManager::registerFileVariables(kCurrentDocumentPrefix, tr("Current document"),
+    globalMacroExpander()->registerFileVariables(kCurrentDocumentPrefix, tr("Current document"),
         []() -> QString {
             IDocument *document = EditorManager::currentDocument();
             return document ? document->filePath() : QString();
         });
 
-    VariableManager::registerIntVariable(kCurrentDocumentXPos,
+    globalMacroExpander()->registerIntVariable(kCurrentDocumentXPos,
         tr("X-coordinate of the current editor's upper left corner, relative to screen."),
         []() -> int {
             IEditor *editor = EditorManager::currentEditor();
             return editor ? editor->widget()->mapToGlobal(QPoint(0, 0)).x() : 0;
         });
 
-    VariableManager::registerIntVariable(kCurrentDocumentYPos,
+    globalMacroExpander()->registerIntVariable(kCurrentDocumentYPos,
         tr("Y-coordinate of the current editor's upper left corner, relative to screen."),
         []() -> int {
             IEditor *editor = EditorManager::currentEditor();

src/plugins/coreplugin/externaltool.cpp

@@ -550,10 +550,12 @@ bool ExternalToolRunner::resolve()
     m_resolvedExecutable.clear();
     m_resolvedArguments.clear();
     m_resolvedWorkingDirectory.clear();
+
+    Utils::MacroExpander *expander = globalMacroExpander();
     { // executable
         QStringList expandedExecutables; /* for error message */
         foreach (const QString &executable, m_tool->executables()) {
-            QString expanded = Core::VariableManager::expandedString(executable);
+            QString expanded = expander->expandedString(executable);
             expandedExecutables << expanded;
             m_resolvedExecutable =
                     Utils::Environment::systemEnvironment().searchInPath(expanded);
@@ -574,14 +576,13 @@ bool ExternalToolRunner::resolve()
         }
     }
     { // arguments
-        m_resolvedArguments = Utils::QtcProcess::expandMacros(m_tool->arguments(),
-                                               Core::VariableManager::macroExpander());
+        m_resolvedArguments = Utils::QtcProcess::expandMacros(m_tool->arguments(), expander);
     }
     { // input
-        m_resolvedInput = Core::VariableManager::expandedString(m_tool->input());
+        m_resolvedInput = expander->expandedString(m_tool->input());
     }
     { // working directory
-        m_resolvedWorkingDirectory = Core::VariableManager::expandedString(m_tool->workingDirectory());
+        m_resolvedWorkingDirectory = expander->expandedString(m_tool->workingDirectory());
     }
     return true;
 }

src/plugins/coreplugin/jsexpander.cpp

@@ -90,7 +90,7 @@ QString JsExpander::evaluate(const QString &expression, QString *errorMessage)
 JsExpander::JsExpander()
 {
     d = new Internal::JsExpanderPrivate;
-    VariableManager::registerPrefix("JS",
+    globalMacroExpander()->registerPrefix("JS",
         QCoreApplication::translate("Core::JsExpander",
                                     "Evaluate simple Javascript statements.\n"
                                     "The statements may not contain '{' nor '}' characters."),

src/plugins/coreplugin/locator/executefilter.cpp

@@ -91,9 +91,9 @@ void ExecuteFilter::accept(LocatorFilterEntry selection) const
         p->m_commandHistory.prepend(value);
 
     bool found;
-    QString workingDirectory = Core::VariableManager::value("CurrentDocument:Path", &found);
+    QString workingDirectory = globalMacroExpander()->value("CurrentDocument:Path", &found);
     if (!found || workingDirectory.isEmpty())
-        workingDirectory = Core::VariableManager::value("CurrentProject:Path", &found);
+        workingDirectory = globalMacroExpander()->value("CurrentProject:Path", &found);
 
     ExecuteData d;
     d.workingDirectory = workingDirectory;

src/plugins/coreplugin/mainwindow.cpp

@@ -120,8 +120,7 @@ MainWindow::MainWindow() :
     m_editorManager(0),
     m_externalToolManager(0),
     m_progressManager(new ProgressManagerPrivate),
-    m_variableManager(new VariableManager),
-    m_jsExpander(new JsExpander), // must be initialized after the VariableManager
+    m_jsExpander(new JsExpander),
     m_vcsManager(new VcsManager),
     m_statusBarManager(0),
     m_modeManager(0),
@@ -305,8 +304,6 @@ MainWindow::~MainWindow()
 
     delete m_helpManager;
     m_helpManager = 0;
-    delete m_variableManager;
-    m_variableManager = 0;
     delete m_jsExpander;
     m_jsExpander = 0;
 }

src/plugins/coreplugin/mainwindow.h

@@ -65,7 +65,6 @@ class ProgressManager;
 class NavigationWidget;
 class RightPaneWidget;
 class SettingsDatabase;
-class VariableManager;
 class VcsManager;
 
 namespace Internal {
@@ -174,7 +173,6 @@ private:
     ExternalToolManager *m_externalToolManager;
     MessageManager *m_messageManager;
     ProgressManagerPrivate *m_progressManager;
-    VariableManager *m_variableManager;
     JsExpander *m_jsExpander;
     VcsManager *m_vcsManager;
     StatusBarManager *m_statusBarManager;

src/plugins/coreplugin/variablechooser.cpp

@@ -67,7 +67,7 @@ public:
         m_variableList = new QListWidget(q);
         m_variableList->setAttribute(Qt::WA_MacSmallSize);
         m_variableList->setAttribute(Qt::WA_MacShowFocusRect, false);
-        foreach (const QByteArray &variable, VariableManager::variables())
+        foreach (const QByteArray &variable, globalMacroExpander()->variables())
             m_variableList->addItem(QString::fromLatin1(variable));
 
         m_variableDescription = new QLabel(q);
@@ -209,8 +209,8 @@ void VariableChooserPrivate::updateDescription(const QString &variable)
     if (variable.isNull())
         m_variableDescription->setText(m_defaultDescription);
     else
-        m_variableDescription->setText(VariableManager::variableDescription(variable.toUtf8())
-            + QLatin1String("<p>") + tr("Current Value: %1").arg(VariableManager::value(variable.toUtf8())));
+        m_variableDescription->setText(globalMacroExpander()->variableDescription(variable.toUtf8())
+            + QLatin1String("<p>") + tr("Current Value: %1").arg(globalMacroExpander()->value(variable.toUtf8())));
 }
 
 /*!

src/plugins/coreplugin/variablemanager.cpp

@@ -30,312 +30,27 @@
 
 #include "variablemanager.h"
 
-#include <utils/stringutils.h>
 
 #include <QCoreApplication>
-#include <QFileInfo>
-#include <QMap>
-#include <QDebug>
 
-static const char kFilePathPostfix[] = ":FilePath";