tag:infosharestrat.logdown.com,2005:/posts„INFOrmation SHAR(E)ing STRATegy”2014-02-13T14:43:53-05:00tag:infosharestrat.logdown.com,2005:Post/1788392014-02-13T14:43:00-05:002016-02-29T12:22:02-05:00A Paradigm Shift in the Information Interoperability / Information Sharing Space<p><em>As background, I was one of the leaders of the <a href="http://www.jnet.pa.gov/portal/server.pt/community/jnet_internet/19236" target="_blank">Pennsylvania Justice Network (JNET)</a> team back in 1997-1999, which first implemented secure XML-formatted and event-drive information exchanges in the justice space that ultimately contributed to the definition of the <a href="https://it.ojp.gov/jxdm/" target="_blank">Global Justice XML Data Model</a> (GJXDM) in 2001 and the <a href="https://www.niem.gov/Pages/default.aspx" target="_blank">National Information Exchange Model</a> (NIEM) <a href="https://www.niem.gov/communities/justice/Pages/about-justice.aspx" target="_blank">Justice domain</a> in 2005. It is time to look forward!</em></p>
<p>A paradigm shift has occurred in the Information Interoperability / Information Sharing space. This shift is the movement from sharing information via exchanging data to finding and sharing it via the linking of data. While technical folks have been building fancier horse and buggy whips, the Henry Fords of the world are about to open their first factory assembly lines delivering Semantic Web Technology (SWT) (see diagram below) based information sharing via Linked Data concepts. I predict these open standards, specifications, and solutions will effectively crank out Model Ts like never seen before--and we all know what was then the fate of the horse and buggy whips trade.</p>
<p></p><p href="http://bnode.org/media/2009/07/08/semantic_web_technology_stack.png"><img alt="" src="http://bnode.org/media/2009/07/08/semantic_web_technology_stack.png" width="900" height="600"> The Semantic Web Technology Stack</p>
<ul>
<li>Click <a title="The Semantic Web Technology Stack" href="https://www.lucidchart.com/documents/view/42dd-3f1c-515dd738-aa12-6d060a000cd9" target="_blank">HERE</a> for an interactive version of this diagram with "hot links" to existing and available <a title="Open RDF Triple Store Solutions" href="http://evectis.com/infosharestrat/open-linkeddata-solutions/rdf-triple-store-solutions/" target="_blank">Resource Description Framework</a> (RDF), <a title="Semantic Web Model Solutions" href="http://evectis.com/infosharestrat/open-linkeddata-solutions/semantic-web-model-solutions/" target="_blank">Models</a>, and <a title="LinkedData Framework Solutions" href="http://evectis.com/infosharestrat/open-linkeddata-solutions/linkeddata-framework-solutions/" target="_blank">Linked Data Stack Framework</a> solutions.</li>
</ul>
<p>It seems to me that the entire "information exchange" community needs to be looking into the future 15 years or so instead of merely tweaking 15+ year old concepts and methods. This paradigm shift is forcing a move “up” the stack as we [and our machines ;)] leverage RDF in direct support of information exchange along with defining more advanced Models and Rules, which ultimately provide support for the intelligent "Logic-based" linkage of <a href="http://en.wikipedia.org/wiki/Federated_database_system" target="_blank">federated data</a> instead of the constant exchanging of unnecessary data.</p>
<ul>
<li>A nice document that helps clarify terminology that is used in the field of linking data is here: <em><a href="https://gforge.inria.fr/docman/view.php/2935/7680/D4.1.pdf" target="_blank">Methods for automated dataset interlinking</a></em>.</li>
</ul>
<h4>My <a title="My Vision for an ENHANCED LINKEDDATA ARCHITECTURE – PART 1" href="http://evectis.com/infosharestrat/2013/04/25/my-vision-for-an-enhanced-linkeddata-architecture-part-1/">future posts</a> introduce a vision for an Enhanced Linked Data Architecture…stay tuned!</h4>
<p>=david.l.woolfenden</p>
davidlwoolfendentag:infosharestrat.logdown.com,2005:Post/7730682016-07-10T15:43:00-04:002016-07-10T15:59:25-04:00TensorFlow<h2>Hello, TensorFlow</h2><h3>A beginner-level, getting started, basic introduction to TensorFlow</h3>
<p>TensorFlow is a general-purpose system for graph-based computation. A typical use is machine learning. In this notebook, we'll introduce the basic concepts of TensorFlow using some simple examples.</p>
<p>TensorFlow gets its name from <a href="https://en.wikipedia.org/wiki/Tensor">tensors</a>, which are arrays of arbitrary dimensionality. A vector is a 1-d array and is known as a 1st-order tensor. A matrix is a 2-d array and a 2nd-order tensor. The "flow" part of the name refers to computation flowing through a graph. Training and inference in a neural network, for example, involves the propagation of matrix computations through many nodes in a computational graph.</p>
<p>When you think of doing things in TensorFlow, you might want to think of creating tensors (like matrices), adding operations (that output other tensors), and then executing the computation (running the computational graph). In particular, it's important to realize that when you add an operation on tensors, it doesn't execute immediately. Rather, TensorFlow waits for you to define all the operations you want to perform. Then, TensorFlow optimizes the computation graph, deciding how to execute the computation, before generating the data. Because of this, a tensor in TensorFlow isn't so much holding the data as a placeholder for holding the data, waiting for the data to arrive when a computation is executed.</p>
<h3>Adding two vectors in TensorFlow</h3>
<p>Let's start with something that should be simple. Let's add two length four vectors (two 1st-order tensors):</p>
<p>$\begin{bmatrix} 1. & 1. & 1. & 1.\end{bmatrix} + \begin{bmatrix} 2. & 2. & 2. & 2.\end{bmatrix} = \begin{bmatrix} 3. & 3. & 3. & 3.\end{bmatrix}$</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>import tensorflow as tf
with tf.Session():
input1 = tf.constant([1.0, 1.0, 1.0, 1.0])
input2 = tf.constant([2.0, 2.0, 2.0, 2.0])
output = tf.add(input1, input2)
result = output.eval()
print result
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[ 3. 3. 3. 3.]
</pre></div>
</figure>
<p>What we're doing is creating two vectors, [1.0, 1.0, 1.0, 1.0] and [2.0, 2.0, 2.0, 2.0], and then adding them. Here's equivalent code in raw Python and using numpy:</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>print [x + y for x, y in zip([1.0] * 4, [2.0] * 4)]
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[3.0, 3.0, 3.0, 3.0]
</pre></div>
</figure>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>import numpy as np
x, y = np.full(4, 1.0), np.full(4, 2.0)
print "{} + {} = {}".format(x, y, x + y)
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[ 1. 1. 1. 1.] + [ 2. 2. 2. 2.] = [ 3. 3. 3. 3.]
</pre></div>
</figure><h3>Details of adding two vectors in TensorFlow</h3>
<p>The example above of adding two vectors involves a lot more than it seems, so let's look at it in more depth.</p>
<blockquote>
<p><code>import tensorflow as tf</code></p>
</blockquote>
<p>This import brings TensorFlow's public API into our IPython runtime environment.</p>
<blockquote>
<p><code>with tf.Session():</code></p>
</blockquote>
<p>When you run an operation in TensorFlow, you need to do it in the context of a <code>Session</code>. A session holds the computation graph, which contains the tensors and the operations. When you create tensors and operations, they are not executed immediately, but wait for other operations and tensors to be added to the graph, only executing when finally requested to produce the results of the session. Deferring the execution like this provides additional opportunities for parallelism and optimization, as TensorFlow can decide how to combine operations and where to run them after TensorFlow knows about all the operations. </p>
<blockquote>
<blockquote>
<p><code>input1 = tf.constant([1.0, 1.0, 1.0, 1.0])</code></p>
<p><code>input2 = tf.constant([2.0, 2.0, 2.0, 2.0])</code></p>
</blockquote>
</blockquote>
<p>The next two lines create tensors using a convenience function called <code>constant</code>, which is similar to numpy's <code>array</code> and numpy's <code>full</code>. If you look at the code for <code>constant</code>, you can see the details of what it is doing to create the tensor. In summary, it creates a tensor of the necessary shape and applies the constant operator to it to fill it with the provided values. The values to <code>constant</code> can be Python or numpy arrays. <code>constant</code> can take an optional shape paramter, which works similarly to numpy's <code>fill</code> if provided, and an optional name parameter, which can be used to put a more human-readable label on the operation in the TensorFlow operation graph.</p>
<blockquote>
<blockquote>
<p><code>output = tf.add(input1, input2)</code></p>
</blockquote>
</blockquote>
<p>You might think <code>add</code> just adds the two vectors now, but it doesn't quite do that. What it does is put the <code>add</code> operation into the computational graph. The results of the addition aren't available yet. They've been put in the computation graph, but the computation graph hasn't been executed yet.</p>
<blockquote>
<blockquote>
<p><code>result = output.eval()</code></p>
<p><code>print result</code></p>
</blockquote>
</blockquote>
<p><code>eval()</code> is also slightly more complicated than it looks. Yes, it does get the value of the vector (tensor) that results from the addition. It returns this as a numpy array, which can then be printed. But, it's important to realize it also runs the computation graph at this point, because we demanded the output from the operation node of the graph; to produce that, it had to run the computation graph. So, this is the point where the addition is actually performed, not when <code>add</code> was called, as <code>add</code> just put the addition operation into the TensorFlow computation graph.</p>
<h3>Multiple operations</h3>
<p>To use TensorFlow, you add operations on tensors that produce tensors to the computation graph, then execute that graph to run all those operations and calculate the values of all the tensors in the graph.</p>
<p>Here's a simple example with two operations:</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>import tensorflow as tf
with tf.Session():
input1 = tf.constant(1.0, shape=[4])
input2 = tf.constant(2.0, shape=[4])
input3 = tf.constant(3.0, shape=[4])
output = tf.add(tf.add(input1, input2), input3)
result = output.eval()
print result
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[ 6. 6. 6. 6.]
</pre></div>
</figure>
<p>This version uses <code>constant</code> in a way similar to numpy's <code>fill</code>, specifying the optional shape and having the values copied out across it.</p>
<p>The <code>add</code> operator supports operator overloading, so you could try writing it inline as <code>input1 + input2</code> instead as well as experimenting with other operators.</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>with tf.Session():
input1 = tf.constant(1.0, shape=[4])
input2 = tf.constant(2.0, shape=[4])
output = input1 + input2
print output.eval()
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[ 3. 3. 3. 3.]
</pre></div>
</figure><h3>Adding two matrices</h3>
<p>Next, let's do something very similar, adding two matrices:</p>
<p>$\begin{bmatrix}</p>
<ol>
<li>& 1. & 1. \</li>
<li>& 1. & 1. \
\end{bmatrix} +
\begin{bmatrix}</li>
<li>& 2. & 3. \</li>
<li>& 5. & 6. \
\end{bmatrix} =
\begin{bmatrix}</li>
<li>& 3. & 4. \</li>
<li>& 6. & 7. \
\end{bmatrix}$</li>
</ol>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>import tensorflow as tf
import numpy as np
with tf.Session():
input1 = tf.constant(1.0, shape=[2, 3])
input2 = tf.constant(np.reshape(np.arange(1.0, 7.0, dtype=np.float32), (2, 3)))
output = tf.add(input1, input2)
print output.eval()
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[[ 2. 3. 4.]
[ 5. 6. 7.]]
</pre></div>
</figure>
<p>Recall that you can pass numpy or Python arrays into <code>constant</code>.</p>
<p>In this example, the matrix with values from 1 to 6 is created in numpy and passed into <code>constant</code>, but TensorFlow also has <code>range</code>, <code>reshape</code>, and <code>tofloat</code> operators. Doing this entirely within TensorFlow could be more efficient if this was a very large matrix.</p>
<p>Try experimenting with this code a bit -- maybe modifying some of the values, using the numpy version, doing this using, adding another operation, or doing this using TensorFlow's <code>range</code> function.</p>
<h3>Multiplying matrices</h3>
<p>Let's move on to matrix multiplication. This time, let's use a bit vector and some random values, which is a good step toward some of what we'll need to do for regression and neural networks.</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>#@test {"output": "ignore"}
import tensorflow as tf
import numpy as np
with tf.Session():
input_features = tf.constant(np.reshape([1, 0, 0, 1], (1, 4)).astype(np.float32))
weights = tf.constant(np.random.randn(4, 2).astype(np.float32))
output = tf.matmul(input_features, weights)
print "Input:"
print input_features.eval()
print "Weights:"
print weights.eval()
print "Output:"
print output.eval()
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>Input:
[[ 1. 0. 0. 1.]]
Weights:
[[-0.8187139 -0.81037313]
[-0.31439888 -2.36761999]
[-1.3127892 -0.33629459]
[-1.23475349 -1.19031894]]
Output:
[[-2.05346727 -2.00069213]]
</pre></div>
</figure>
<p>Above, we're taking a 1 x 4 vector [1 0 0 1] and multiplying it by a 4 by 2 matrix full of random values from a normal distribution (mean 0, stdev 1). The output is a 1 x 2 matrix.</p>
<p>You might try modifying this example. Running the cell multiple times will generate new random weights and a new output. Or, change the input, e.g., to [0 0 0 1]), and run the cell again. Or, try initializing the weights using the TensorFlow op, e.g., <code>random_normal</code>, instead of using numpy to generate the random weights.</p>
<p>What we have here is the basics of a simple neural network already. If we are reading in the input features, along with some expected output, and change the weights based on the error with the output each time, that's a neural network.</p>
<h3>Use of variables</h3>
<p>Let's look at adding two small matrices in a loop, not by creating new tensors every time, but by updating the existing values and then re-running the computation graph on the new data. This happens a lot with machine learning models, where we change some parameters each time such as gradient descent on some weights and then perform the same computations over and over again.</p>
<figure class="figure-code code"><figcaption><span>
</span></figcaption><div class="highlight"><pre>#@test {"output": "ignore"}
import tensorflow as tf
import numpy as np
with tf.Session() as sess:
# Set up two variables, total and weights, that we'll change repeatedly.
total = tf.Variable(tf.zeros([1, 2]))
weights = tf.Variable(tf.random_uniform([1,2]))
# Initialize the variables we defined above.
tf.initialize_all_variables().run()
# This only adds the operators to the graph right now. The assignment
# and addition operations are not performed yet.
update_weights = tf.assign(weights, tf.random_uniform([1, 2], -1.0, 1.0))
update_total = tf.assign(total, tf.add(total, weights))
for _ in range(5):
# Actually run the operation graph, so randomly generate weights and then
# add them into the total. Order does matter here. We need to update
# the weights before updating the total.
sess.run(update_weights)
sess.run(update_total)
print weights.eval(), total.eval()
</pre></div>
</figure>
<figure class="figure-code code"><div class="highlight"><pre>[[-0.41494703 0.47648168]] [[-0.41494703 0.47648168]]
[[ 0.35746408 0.99504066]] [[-0.05748296 1.47152233]]
[[-0.46462393 -0.80201006]] [[-0.52210689 0.66951227]]
[[-0.99513483 -0.42322445]] [[-1.51724172 0.24628782]]
[[ 0.13371086 -0.85545826]] [[-1.38353086 -0.60917044]]
</pre></div>
</figure>
<p>This is more complicated. At a high level, we create two variables and add operations over them, then, in a loop, repeatedly execute those operations. Let's walk through it step by step.</p>
<p>Starting off, the code creates two variables, <code>total</code> and <code>weights</code>. <code>total</code> is initialized to [0, 0] and <code>weights</code> is initialized to random values between -1 and 1.</p>
<p>Next, two assignment operators are added to the graph, one that updates weights with random values from [-1, 1], the other that updates the total with the new weights. Again, the operators are not executed here. In fact, this isn't even inside the loop. We won't execute these operations until the <code>eval</code> call inside the loop.</p>
<p>Finally, in the for loop, we run each of the operators. In each iteration of the loop, this executes the operators we added earlier, first putting random values into the weights, then updating the totals with the new weights. This call uses <code>eval</code> on the session; the code also could have called <code>eval</code> on the operators (e.g. <code>update_weights.eval</code>).</p>
<p>It can be a little hard to wrap your head around exactly what computation is done when. The important thing to remember is that computation is only performed on demand.</p>
<p>Variables can be useful in cases where you have a large amount of computation and data that you want to use over and over again with just a minor change to the input each time. That happens quite a bit with neural networks, for example, where you just want to update the weights each time you go through the batches of input data, then run the same operations over again.</p>
<h3>What's next?</h3>
<p>This has been a gentle introduction to TensorFlow, focused on what TensorFlow is and the very basics of doing anything in TensorFlow. If you'd like more, the next tutorial in the series is Getting Started with TensorFlow, also available in there - <a href="http://vps70137.vps.ovh.ca:8888/tree" rel="nofollow" target="_blank">http://vps70137.vps.ovh.ca:8888/tree</a> .</p>
davidlwoolfendentag:infosharestrat.logdown.com,2005:Post/1925672014-04-05T10:55:00-04:002016-02-29T12:24:29-05:00Direct link to the most recent post<p><a href="http://evectis.com/infosharestrat/" title="The most recent post.....">http://evectis.com/infosharestrat/</a></p>
davidlwoolfenden