This is an oldie but a goodie, so I'll re-post it here on this new blog.
How do you read Java properties files with shell? Maybe another good question is:
Why would you want to read Java properties files with shell? Let's start with that.
Why?
Quite a few Java Enterprise systems that I've worked on used Java (ANT) properties files for build and runtime configuration. It's an easy to edit and easy to read for both humans and Java programs. However, when running Java Enterprise on Linux, you invariably get some shell scripts for process control, deployment, etc. These need to know about database host names, and other things that are included in the configuration file.
Method 1 - Scan through the file to get an individual property
This works well if you just want to read a
specific property in a Java-style properties file. Here's what it does:
- Remove comments (lines that start with '#')
- Grep for the property
- Get the last value using
tail -n 1
- Strip off the property name and the '=' (using cut -d "=" -f2-) to handle values with '=' in them, as a commenter suggested).
sed '/^\#/d' myprops.properties | grep 'someproperty' | tail -n 1 | cut -d "=" -f2-
It can also be handy to strip off the leading and trailing blanks with this sed expression:
s/^[[:space:]]*//;s/[[:space:]]*$//
Which makes the whole thing look like this:
sed '/^\#/d' myprops.properties | grep 'someproperty' | tail -n 1 | cut -d "=" -f2- | sed 's/^[[:space:]]*//;s/[[:space:]]*$//'
Shell scripts can use this technique to set environment variables, for example:
JAVA_HOME=`sed '/^\#/d' build.properties | grep 'jdk.home' | tail -n 1 | cut -d "=" -f2- | sed 's/^[[:space:]]*//;s/[[:space:]]*$//'`
Method 2 - Convert the properties file to a shell script, then run it
This method is good for when you want to read the whole properties file.
However, it only works if the property names are valid shell variable
names. For example, property names with a '.' in them will cause this to
break.
Here is the original article from
+Michael Slinn that inspired this method.
Security Note
These techniques generate shell code from the properties file, then execute the generated code. This can be a potential security issue if the properties file has hidden shell code in it. Make sure you have appropriate access control on the properties file in this case.
The steps are:
- Create a temp file.
- Stream the properties file, convert it to unix LF format (unless you are sure the file was produced on Linux).
- (not sure what the first 'sed' does)
- Double quote all the values, to preserve spaces.
- Source in the file. Shell should ignore Java properties file comments automatically.
- Erase the temp file.
Here is the code:
TEMPFILE=$(mktemp)
cat somefile.properties | dos2unix |
sed -re 's/"/"/'g|sed -re 's/=(.*)/="\1"/g'>$TEMPFILE
source $TEMPFILE
rm $TEMPFILE
Method 2 reloaded - Use AWK
This method is the same as the previous one except we use an inline
awk
script to replace any non-alpha-digit-underscore characters in the
property names with underscore. This solves the problem the previous
method has when it processes properties that have '.' in the name, for
example.
Some clever person could probably figure out how to do the same thing with
sed, but
awk works.
Steps:
- Create a temporary file.
- Stream the properties file to stdout and (optionally) filter out dos style linefeeds.
- Split
each line using "=" as the field separator. If there are exactly two
fields, then substitute the non-identifier characters with underscore in
the first one, and print the second field with quotes around it.
- If the line doesn't have an =, or seems to have more than two values, just print it out.
- Source in the temporary file and delete it.
Teh codez:
TEMPFILE=$(mktemp)
cat somefile.properties | dos2unix |
awk 'BEGIN { FS="="; } \
/^\#/ { print; } \
!/^\#/ { if (NF >= 2) { n = $1; gsub(/[^A-Za-z0-9_]/,"_",n); print n "=\"" $2 "\""; } else { print; } }' \
>$TEMPFILE
source $TEMPFILE
rm $TEMPFILE
Method 2 - Using Groovy
This is my current favorite! Since Groovy has all of the ANT facilities available, it can do all the property interpolation things that ANT does, plus handle all the quirks of Java Properties files that the other methods may not handle easily.
The example here can read two properties files: one for default values, and one for overrides that are environment specific. This is great for when you have a lot of values that are the same most of the time, and only a few which change based on the environment.
#!/usr/bin/groovy
def generateEnvVars(File propertiesFile,File defaultsFile) {
println "#!/bin/bash"
AntBuilder ant = new AntBuilder()
print "# Reading ${propertiesFile} ..."
ant.property (file: propertiesFile)
println "OK"
if (defaultsFile) {
print "# Reading ${defaultsFile} ..."
ant.property (file: defaultsFile)
println "OK"
}
// The properties are now in ant.project.properties
def props = ant.project.properties
props.keySet().toList().sort().each { String name ->
// Shell-ify the property name.
def shellVar = name.replaceAll(/[^A-Za-z0-9_]/, '_')
def value = props[name]
println "${shellVar}=\'${value}\'"
}
}
// Main
// args[0] - Name of the main properties file
// args[1] (optional) - Name of the defaults properties file
generateEnvVars(new File(args[0]), args.length > 1 ? new File(args[1]) : null)
This reads the properties file (the first argument) and optionally the default properties file (the second argument), and writes the shell statements to stdout. Since we're using ANT to read in the properties files, this version has a lot more functionality:
- All the standard ANT properties will be processed.
- ANT will interpolate the property values.
So, if we have the following properties file:
# properties example
some.property=this is a property
multiline.property=this is a java \
multi line property
interpolated=the value of some.property is "${some.property}"
The generated shell script will contain:
some_property='this is a property'
multiline_property='this is a java multi line property'
interpolated='the value of some.property is "this is a property"'
Edge Case - Shell commands in property values
What happens when the properties file looks like this?
zzzhack=' echo "all your base are belong to us" '
When we use the Groovy script with this file, we hit an edge case:
$ groovy readprops.groovy my.properties > env-vars.sh
$ source env-vars.sh
all your base are belong to us
Uh oh! This property value could have done something very bad. We can change the Groovy script to escape any single quotes like this:
def value = props[name].replaceAll(/[\']/,"\\\\'")
However, this may break some things. Of course, this isn't a problem if you don't embed shell commands into your properties files... with single quotes around them. (yeah... well... maybe you shouldn't do that!)
Here's the previous generation of this post:
http://shrubbery.homeip.net/c/display/W/Reading+Java-style+Properties+Files+with+Shell
