并行编程实战——TBB框架的应用之三Supra的配置文件管理
一、Supra配置文件
其实这一段和TBB本身的联系并不大，讲它略有些鸡肋。但是不分析它又不好分析后面的节点的动态管理，所以还是拉过来一起分析一下。在前面其实简单介绍过这一个模块，它主要是通过配置文件的读写操作来动态处理节点的参数传递。同时，在Supra中支持实时的对参数的修改，这个显得尤为突出。
其配置管理文件主要为XML类型，在XML的节点中，定义上节点的名称、类型和ID。并且在其子项中定义了相关的参数及参数的值。而在Supra程序内自动通过tinyxml调用来解析这个配置文件获取得相关的信息（包括但不限于上面提到的参数等）。配置文件中还支持定义Supra类型节点的连接状态的定义和相关参数处理，这样就可以直接进行整个项目中节点的数据流向和分析的控制。
二、Supra对配置文件的管理
Supra中对配置文件的管理分成两大块，即节点的管理和配置的动态处理。
1、节点的管理
节点的管理类似于静态的定义节点的名称等用来生成实例和进行节点间的连接。在程序中基本分为普通节点的读写、输入节点的读写和输出节点的读写以及后面的连接控制读写操作。这样就形成了整体认知上的静态管理。
2、配置的动态处理
这个更多的是配置文件选项的自我管理，提供了不同数据类型的处理及相关参数发生变化后动态通知程序进行更新。
三、动态管理配置文件
节点的动态加载这一块基本已经分析过了，如果想要查看更细节的代码可以下载其源码进行分析即可。这里重点分析一下其对配置文件的动态管理这部分。从应用层次上看，其为分三个层次：
1、节点应用
//抽象基类
//This is ugly, but "queueing" and "rejecting" are enum-values in old versions and types in newer versions...
#if TBB_INTERFACE_VERSION_MAJOR >= 9
#define TBB_QUEUE_RESOLVER(__buffering__) typename std::conditional<__buffering__, tbb::flow::queueing, tbb::flow::rejecting>::type
#else
#define TBB_QUEUE_RESOLVER(__buffering__) (__buffering__ ? tbb::flow::queueing : tbb::flow::rejecting)
#endif //TBB_INTERFACE_VERSION_MAJOR >= 9
namespace supra
{
class RecordObject;
}
namespace supra
{
/*! \brief Abstract interface for a general node (input, output or processing).
*
* This is the common interface for all nodes.
*/
class AbstractNode
{
......
public:
/// Base constructor for all nodes
AbstractNode(const std::string & nodeID, bool queueing)
: m_nodeID(nodeID)
, m_queueing(queueing)
{
m_configurationDictionary.setValueRangeDictionary(&m_valueRangeDictionary);
}
......
/// Returns a const pointer to the \see ValueRangeDictionary of this node
/// The ValueRangeDictionary describes the parameters of the node and
/// their valid ranges
const ValueRangeDictionary * getValueRangeDictionary()
{
return &m_valueRangeDictionary;
}
/// Returns a const pointer to the \see ConfigurationDictionary of this node
/// The ConfigurationDictionary contains the parameters currently set
/// and their values
const ConfigurationDictionary * getConfigurationDictionary()
{
return &m_configurationDictionary;
}
/// Returns the ID of the node.
/// Node IDs are unique and cannot be changed after creation.
const std::string& getNodeID()
{
return m_nodeID;
}
/// Templated interface to change a node parameter
/// returns whether the value was valid
template<typename ValueType>
bool changeConfig(const std::string& configKey, const ValueType& newValue)
{
if (m_valueRangeDictionary.hasKey(configKey) &&
m_valueRangeDictionary.isInRange(configKey, newValue))
{
logging::log_parameter("Parameter: ", m_nodeID, ".", configKey, " = ", newValue);
m_configurationDictionary.set(configKey, newValue);
configurationEntryChanged(configKey);
return true;
}
return false;
}
/// Function to set the whole \see ConfiguraitonDictionary at once
/// Only parameters whose value is valid are applied
void changeConfig(const ConfigurationDictionary& newConfig)
{
// trigger callback for major configuration changes in overloaded implementation
configurationDictionaryChanged(newConfig);
//validate the configuration entries
ConfigurationDictionary validConfig = newConfig;
validConfig.setValueRangeDictionary(&m_valueRangeDictionary);
validConfig.checkEntriesAndLog(m_nodeID);
//store all valid entries
m_configurationDictionary = validConfig;
configurationChanged();
}
/// Returns a string with the timing info (call frequency and run-time)
/// if the node uses the \see CallFrequency member to monitor itself
std::string getTimingInfo()
{
return m_callFrequency.getTimingInfo();
}
protected:
/// The collection of node parameters
ConfigurationDictionary m_configurationDictionary;
/// The definition of parameters and their respective ranges
ValueRangeDictionary m_valueRangeDictionary;
/// \see CallFrequency can be used by the node implementation to monitor its
/// timings (frequency of activation and run-time)
CallFrequency m_callFrequency;
bool m_queueing;
protected:
/// Callback for the node implementation to be notified of the change of parameters.
/// Needs to be overwritten and thread-safe
virtual void configurationEntryChanged(const std::string& configKey) {};
/// Callback for the node implementation to be notified of the change of parameters.
/// Needs to be overwritten and thread-safe
virtual void configurationChanged() {};
/// Callback for the node implementation to be notified of the change of a full dictionary change.
/// can be be overwritten but must be thread-safe
/// dictionary does not contain fail-safe mechanisms yet, thus should be only used to adjust value range changes or add new settings on the fly
virtual void configurationDictionaryChanged(const ConfigurationDictionary& newConfig) {};
private:
std::string m_nodeID;
};
}
#endif //!__ABSTRACTNODE_H__
InputDevice::InputDevice(tbb::flow::graph& graph, const std::string & nodeID)
: AbstractInput(graph, nodeID,1)
, m_sequenceIndex(0)
, m_frameIndex(0)
, m_numel(0)
, m_frozen(false)
, m_lastFrame(false)
, m_ready(false)
{
m_callFrequency.setName("RawMhd");
//Setup allowed values for parameters
m_valueRangeDictionary.set<bool>("singleImage", { true, false }, false, "Single image");
m_valueRangeDictionary.set<bool>("streamSequenceOnce", { true, false }, false, "Emit sequences once");
m_valueRangeDictionary.set<double>("frequency", 0.001, 100, 5, "Frequency");
m_valueRangeDictionary.set<std::string>("mhdMetaDataFilename", "", "User meta data filename");
m_valueRangeDictionary.set<std::string>("mhdDataFilename", "", "User data filename");
readConfiguration();
}
void InputDevice::configurationEntryChanged(const std::string& configKey)
{
std::lock_guard<std::mutex> lock(m_objectMutex);
if (configKey == "frequency")
{
m_frequency = m_configurationDictionary.get<double>("frequency");
if (getTimerFrequency() != m_frequency)
{
setUpTimer(m_frequency);
}
}
if (configKey == "singleImage")
{
m_singleImage = m_configurationDictionary.get<bool>("singleImage");
}
if (configKey == "streamSequenceOnce")
{
m_streamSequenceOnce = m_configurationDictionary.get<bool>("streamSequenceOnce");
}
}
void InputDevice::configurationChanged()
{
this->readConfiguration();
}
上面是一个输入节点的配置文件的设置。可以在构造函数或者初始化函数中对配置项进行修改或者设置初始值，然后在节点变化后调用Change相关函数即可。因为这个类继承自抽象基类，所以把相关的配置动作函数和变量贴了上来。如果对读写配置文件的函数还有印象，里面就有类似“in->changeConfig(dict);”之类的代码。
2、配置字典
#ifndef __CONFIGURATIONDICTIONARY_H__
#define __CONFIGURATIONDICTIONARY_H__
#include <string>
#include <memory>
#include <tuple>
#include <map>
#include "utilities/tinyxml2/tinyxml2.h"
#include "utilities/TemplateTypeDefault.h"
#include <utilities/Logging.h>
#include "ValueRangeDictionary.h"
namespace supra
{
class ConfigurationEntryType
{
public:
virtual ~ConfigurationEntryType() {}
};
template <typename ValueType>
class ConfigurationEntry : public ConfigurationEntryType
{
public:
ConfigurationEntry(ValueType value) : m_valueEntry(value) {}
virtual ~ConfigurationEntry() {}
virtual const ValueType& get() const { return m_valueEntry; }
private:
ValueType m_valueEntry;
};
class ConfigurationDictionary
{
public:
ConfigurationDictionary()
:p_valueRangeDictionary(nullptr) {};
ConfigurationDictionary(const tinyxml2::XMLElement* parentXmlElement);
const ConfigurationDictionary& operator=(const ConfigurationDictionary& a)
{
m_mapEntries = a.m_mapEntries;
return *this;
}
void setValueRangeDictionary(const ValueRangeDictionary* valueRangeDictionary)
{
p_valueRangeDictionary = valueRangeDictionary;
}
void toXml(tinyxml2::XMLElement* parentXmlElement) const;
template <typename ValueType>
void set(const std::string& key, const ValueType& value) {
auto valueEntry = std::make_shared<ConfigurationEntry<ValueType> >(value);
m_mapEntries[key] = valueEntry;
}
template <typename ValueType>
ValueType get(const std::string& key, const ValueType& defaultValue) const {
auto iteratorValue = m_mapEntries.find(key);
if (iteratorValue != m_mapEntries.end())
{
auto pValueEntry = iteratorValue->second;
const ConfigurationEntry<ValueType> * pValueTyped = dynamic_cast<const ConfigurationEntry<ValueType> *>(pValueEntry.get());
if (pValueTyped)
{
return pValueTyped->get();
}
else if (p_valueRangeDictionary)
{
return p_valueRangeDictionary->getDefaultValue<ValueType>(key);
}
else
{
return defaultValue;
}
}
else if (p_valueRangeDictionary && p_valueRangeDictionary->hasKey(key))
{
return p_valueRangeDictionary->getDefaultValue<ValueType>(key);
}
else
{
return defaultValue;
}
}
template <typename ValueType>
ValueType get(const std::string& key) const {
auto iteratorValue = m_mapEntries.find(key);
if (iteratorValue != m_mapEntries.end())
{
auto pValueEntry = iteratorValue->second;
const ConfigurationEntry<ValueType> * pValueTyped = dynamic_cast<const ConfigurationEntry<ValueType> *>(pValueEntry.get());
if (pValueTyped)
{
return pValueTyped->get();
}
else if (p_valueRangeDictionary)
{
return p_valueRangeDictionary->getDefaultValue<ValueType>(key);
}
else
{
logging::log_error("Trying to access parameter '", key, "' without value and valueRangeDictionary");
return TemplateTypeDefault<ValueType>::getDefault();
}
}
else if (p_valueRangeDictionary)
{
return p_valueRangeDictionary->getDefaultValue<ValueType>(key);
}
else
{
logging::log_error("Trying to access parameter '", key, "' without value and valueRangeDictionary");
return TemplateTypeDefault<ValueType>::getDefault();
}
}
void checkEntriesAndLog(const std::string & nodeID)
{
if (p_valueRangeDictionary)
{
std::vector<std::string> toRemove;
for (auto entry : m_mapEntries)
{
bool valueGood = false;
if (p_valueRangeDictionary->hasKey(entry.first))
{
switch (p_valueRangeDictionary->getType(entry.first))
{
case TypeBool:
valueGood = checkEntryAndLogTemplated<bool>(entry.first, nodeID);
break;
case TypeInt8:
valueGood = checkEntryAndLogTemplated<int8_t>(entry.first, nodeID);
break;
case TypeUint8:
valueGood = checkEntryAndLogTemplated<uint8_t>(entry.first, nodeID);
break;
case TypeInt16:
valueGood = checkEntryAndLogTemplated<int16_t>(entry.first, nodeID);
break;
case TypeUint16:
valueGood = checkEntryAndLogTemplated<uint16_t>(entry.first, nodeID);
break;
case TypeInt32:
valueGood = checkEntryAndLogTemplated<int32_t>(entry.first, nodeID);
break;
case TypeUint32:
valueGood = checkEntryAndLogTemplated<uint32_t>(entry.first, nodeID);
break;
case TypeInt64:
valueGood = checkEntryAndLogTemplated<int64_t>(entry.first, nodeID);
break;
case TypeUint64:
valueGood = checkEntryAndLogTemplated<uint64_t>(entry.first, nodeID);
break;
case TypeFloat:
valueGood = checkEntryAndLogTemplated<float>(entry.first, nodeID);
break;
case TypeDouble:
valueGood = checkEntryAndLogTemplated<double>(entry.first, nodeID);
break;
case TypeString:
valueGood = checkEntryAndLogTemplated<std::string>(entry.first, nodeID);
break;
case TypeDataType:
valueGood = checkEntryAndLogTemplated<DataType>(entry.first, nodeID);
break;
case TypeUnknown:
default:
logging::log_error("cannot check validity of configuration entry '", entry.first, "' as its range type is unknown.");
}
}
if (!valueGood)
{
toRemove.push_back(entry.first);
}
}
//remove all determined bad values
for (std::string keyToRemove : toRemove)
{
m_mapEntries.erase(keyToRemove);
logging::log_warn("Removed configuration entry '", keyToRemove, "' because it's type or value were not as defined by the range.");
}
}
else
{
logging::log_error("Configuration checking failed, due to missing valueRangeDictionary");
//we cannot verify any entry. Remove all!
m_mapEntries.clear();
}
}
template <typename ValueType>
bool checkEntryAndLogTemplated(const std::string& key, const std::string & nodeID)
{
auto iteratorValue = m_mapEntries.find(key);
if (iteratorValue != m_mapEntries.end())
{
auto pValueEntry = iteratorValue->second;
const ConfigurationEntry<ValueType> * pValueTyped = dynamic_cast<const ConfigurationEntry<ValueType> *>(pValueEntry.get());
if (pValueTyped)
{
bool entryGood = p_valueRangeDictionary->isInRange<ValueType>(key, pValueTyped->get());
if (entryGood)
{
logging::log_parameter("Parameter: ", nodeID, ".", key, " = ", pValueTyped->get());
}
else {
logging::log_log("Rejected out-of-range parameter: ", nodeID, ".", key, " = ", pValueTyped->get());
}
return entryGood;
}
else
{
//Types do not fit
logging::log_log("Rejected parameter of wrong type for: ", nodeID, ".", key);
return false;
}
}
else {
//The key is not even present
logging::log_log("Rejected unknown parameter: ", nodeID, ".", key);
return false;
}
}
private:
std::map<std::string, std::shared_ptr<ConfigurationEntryType> > m_mapEntries;
const ValueRangeDictionary* p_valueRangeDictionary;
};
}
#endif //!__CONFIGURATIONDICTIONARY_H__
在配置字典里可以看到使用的tinyxml，然后内部对支持的数据类型和各种转化进行了处理。
3、数据类型字典
namespace supra
{
/// Base class for the templated \see ValueRangeEntry
class ValueRangeType
{
public:
virtual ~ValueRangeType() {}
/// Returns the type of the parameter described by the value range
virtual DataType getType() const {
return TypeUnknown;
};
};
/// Describes a node parameter with its valid range. Is part of the parameter system.
/// There are three types of ranges:
/// -Discrete: Only values that are in a fixed vector are valid
/// -Closed Range: Values between the defined upper and lower bound are valid
/// -Unrestricted: All values are valid
/// This is selected on construction by the choice of constructor
template <typename ValueType>
class ValueRangeEntry : public ValueRangeType
{
public:
/// Constructor for a discrete range, takes a vector of the allowed values.
/// Additionally takes the parameter's default value and its display name
ValueRangeEntry(const std::vector<ValueType>& discreteRange, const ValueType& defaultValue, const std::string& displayName)
: m_discreteRange(discreteRange)
, m_continuousRange()
, m_isContinuous(false)
, m_isUnrestricted(false)
, m_defaultValue(defaultValue)
, m_displayName(displayName) {}
/// Constructor for a closed range, takes the lower and upper bound of the allowed values.
/// Additionally takes the parameter's default value and its display name
ValueRangeEntry(const ValueType& lowerBound, const ValueType& upperBound, const ValueType& defaultValue, const std::string& displayName)
: m_discreteRange()
, m_continuousRange({ lowerBound, upperBound })
, m_isContinuous(true)
, m_isUnrestricted(false)
, m_defaultValue(defaultValue)
, m_displayName(displayName) {}
/// Constructor for an unrestricted range,
/// takes the parameter's default value and its display name
ValueRangeEntry(const ValueType& defaultValue, const std::string& displayName)
: m_discreteRange()
, m_continuousRange()
, m_isContinuous(false)
, m_isUnrestricted(true)
, m_defaultValue(defaultValue)
, m_displayName(displayName) {}
virtual ~ValueRangeEntry() {}
/// Return the type of the parameter range
virtual DataType getType() const { return DataTypeGet<ValueType>(); }
/// Returns whether the range is unrestricted
virtual bool isUnrestricted() const { return m_isUnrestricted; }
/// Returns whether the range is closed but continous
virtual bool isContinuous() const { return m_isContinuous; }
/// Returns the allowed values of this range. Should only be called for discrete ranges.
virtual const std::vector<ValueType>& getDiscrete() const { return m_discreteRange; }
/// Returns the upper and lower bound of this range. Should only be called for closed ranges.
virtual const std::pair<ValueType, ValueType>& getContinuous() const { return m_continuousRange; }
/// Returns this parameter's default value
virtual const ValueType & getDefaultValue() const { return m_defaultValue; }
/// Checks whether the value is within the range, that is whether it is valid
virtual bool isInRange(const ValueType& valueToCheck) const {
bool valIsInRange = false;
if (m_isUnrestricted)
{
valIsInRange = true;
}
else if (m_isContinuous)
{
valIsInRange =
valueToCheck >= m_continuousRange.first &&
valueToCheck <= m_continuousRange.second;
}
else {
valIsInRange =
std::find(
m_discreteRange.begin(),
m_discreteRange.end(),
valueToCheck)
!= m_discreteRange.end();
}
return valIsInRange;
}
/// Returns the display name of the parameter
const std::string& getDisplayName() const { return m_displayName; }
private:
bool m_isUnrestricted;
bool m_isContinuous;
std::vector<ValueType> m_discreteRange;
std::pair<ValueType, ValueType> m_continuousRange;
ValueType m_defaultValue;
std::string m_displayName;
};
/// A collection of parameter ranges \see ValueRangeEntry, every node has one
class ValueRangeDictionary
{
public:
/// Default constructor
ValueRangeDictionary() {};
/// Assignment operator. Copies all ranges defined in the assignee
const ValueRangeDictionary& operator=(const ValueRangeDictionary& a)
{
m_mapEntries = a.m_mapEntries;
return *this;
}
/// Creates a discrete range for the given parameter, takes a vector of the allowed values.
/// Additionally takes the parameter's default value and its display name
template <typename ValueType>
void set(const std::string& key, const std::vector<ValueType>& value, const ValueType& defaultValue, const std::string& displayName) {
auto valueEntry = std::make_shared<ValueRangeEntry<ValueType> >(value, defaultValue, displayName);
m_mapEntries[key] = valueEntry;
}
/// Creates a closed range for the given parameter, takes the lower and upper bound of the allowed values.
/// Additionally takes the parameter's default value and its display name
template <typename ValueType>
void set(const std::string& key, const ValueType& lowerBound, const ValueType& upperBound, const ValueType& defaultValue, const std::string& displayName) {
auto valueEntry = std::make_shared<ValueRangeEntry<ValueType> >(lowerBound, upperBound, defaultValue, displayName);
m_mapEntries[key] = valueEntry;
}
/// Creates an unrestricted rangefor the given parameter,
/// takes the parameter's default value and its display name
template <typename ValueType>
void set(const std::string& key, const ValueType& defaultValue, const std::string& displayName) {
auto valueEntry = std::make_shared<ValueRangeEntry<ValueType> >(defaultValue, displayName);
m_mapEntries[key] = valueEntry;
}
/// Checks whether a range for a parameter is defined in this dictionary
bool hasKey(const std::string& key) const
{
return m_mapEntries.find(key) != m_mapEntries.end();
}
/// Returns a list of the parameter ids that are defined in this dictionary
std::vector<std::string> getKeys() const
{
std::vector<std::string> keys(m_mapEntries.size());
std::transform(m_mapEntries.begin(), m_mapEntries.end(), keys.begin(),
[](std::pair<std::string, std::shared_ptr<ValueRangeType> > mapPair) -> std::string {return mapPair.first; });
return keys;
}
/// Returns the type of a parameter. Only defined if `hasKey(key) == true`
DataType getType(std::string key) const
{
auto iteratorValue = m_mapEntries.find(key);
return iteratorValue->second->getType();
}
/// Returns the range for a parameter
template <typename ValueType>
const ValueRangeEntry<ValueType> * get(const std::string& key) const {
auto iteratorValue = m_mapEntries.find(key);
auto pValueEntry = iteratorValue->second;
const ValueRangeEntry<ValueType> * pValueTyped = dynamic_cast<const ValueRangeEntry<ValueType> *>(pValueEntry.get());
return pValueTyped;
}
void remove(const std::string& key) {
m_mapEntries.erase(key);
}
/// Returns the default value of a parameter.
/// If the parameter has no range in this dictionary, the types default values is returned
/// ( \see TemplateTypeDefault)
template <typename ValueType>
ValueType getDefaultValue(const std::string& key) const {
auto iteratorValue = m_mapEntries.find(key);
if (iteratorValue == m_mapEntries.end())
{
return TemplateTypeDefault<ValueType>::getDefault();
}
auto pValueEntry = iteratorValue->second;
const ValueRangeEntry<ValueType> * pValueTyped = dynamic_cast<const ValueRangeEntry<ValueType> *>(pValueEntry.get());
if (pValueTyped)
{
return pValueTyped->getDefaultValue();
}
else {
return TemplateTypeDefault<ValueType>::getDefault();
}
}
/// Checks whether the value is within the range, that is whether it is valid
/// Returns false, if the parameter is not defined in this dictionary
template <typename ValueType>
bool isInRange(const std::string& key, const ValueType& value) const
{
bool result = false;
auto iteratorValue = m_mapEntries.find(key);
if (iteratorValue != m_mapEntries.end())
{
auto pValueEntry = iteratorValue->second;
const ValueRangeEntry<ValueType> * pValueTyped = dynamic_cast<const ValueRangeEntry<ValueType> *>(pValueEntry.get());
result = pValueTyped->isInRange(value);
}
return result;
}
private:
std::map<std::string, std::shared_ptr<ValueRangeType> > m_mapEntries;
};
}
这块代码看上去复杂，其实细看非常简单，就是数据类型的处理和一些配置项的读写操作。之所以看上去有点扎眼就是它用了模板，条条框框比较多，可能不太熟悉相关的业务就会感到不知所措，没关系，多看看，调试一下程序就好了。
通过上述三个部分的联动，就可以进行配置文件的自动读写配置和实时的配置更新了。
四、总结
一个好汉三个帮。完成一个并行任务，也需要一些辅助的模块共同作用。这种利用配置文件来进行节点管理的方法，虽然谈不上多新奇，但成熟可靠，正好应用在医学领域。