Skip to content

Commit

Permalink
Merge pull request opencog#8 from ngeiswei/merge-opencog-to-singnet
Browse files Browse the repository at this point in the history
Merge opencog -> singnet
  • Loading branch information
ngeiswei authored Aug 7, 2020
2 parents a353bfd + 9bc15f7 commit ddc8966
Show file tree
Hide file tree
Showing 14 changed files with 931 additions and 39 deletions.
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ history, despite being deeply flawed).
deleting them, and rates for setting and getting TruthValues and
IncomingSets.

* __micro__ -- Micro-benchmarks for misc items.
* __micro__ -- Micro-benchmarks for misc items. **These benchmarks
do not work correctly! They're just plain broken! They report
incorrect results! DO NOT USE!** See the README in the subdirectory
for details about what's wrong.

* __python__ -- Benchmarks for the python bindings to various AtomSpace
subsystems.
Expand Down
8 changes: 5 additions & 3 deletions atomspace/AtomSpaceBenchmark.cc
Original file line number Diff line number Diff line change
Expand Up @@ -478,15 +478,15 @@ void AtomSpaceBenchmark::startBenchmark(int numThreads)
else {
asp = new AtomSpace();
#if HAVE_CYTHON
pyev = new PythonEval(asp);
pyev = new PythonEval();
// And now ... create a Python instance of the atomspace.
// Pass in the raw C++ atomspace address into cython.
// Kind-of tacky, but I don't see any better way.
// (We must do this because otherwise, the benchmark would
// run on a different atomspace, than the one containing
// all the atoms. And that would give bad results.
std::ostringstream dss;
dss << "from opencog.atomspace import AtomSpace, types, TruthValue, Atom" << std::endl;
dss << "from atomspace import AtomSpace, types, TruthValue, Atom" << std::endl;
dss << "aspace = AtomSpace(" << asp << ")" << std::endl;
pyev->eval(dss.str());
#endif
Expand Down Expand Up @@ -597,10 +597,12 @@ Type AtomSpaceBenchmark::randomType(Type t)
} while (!nameserver().isA(candidateType, t) or
nameserver().isA(candidateType, BOOLEAN_LINK) or
nameserver().isA(candidateType, FREE_LINK) or
nameserver().isA(candidateType, TYPE_INPUT_LINK) or
nameserver().isA(candidateType, TYPE_OUTPUT_LINK) or
nameserver().isA(candidateType, NUMERIC_LINK) or
nameserver().isA(candidateType, SCOPE_LINK) or
nameserver().isA(candidateType, TYPE_LINK) or
nameserver().isA(candidateType, UNIQUE_LINK) or
nameserver().isA(candidateType, TYPED_VARIABLE_LINK) or
candidateType == VARIABLE_LIST or
candidateType == VARIABLE_SET or
candidateType == DEFINE_LINK or
Expand Down
6 changes: 4 additions & 2 deletions atomspace/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ IF (HAVE_CYTHON)
${PYTHON_INCLUDE_DIRS}
)
TARGET_LINK_LIBRARIES (atomspace_bm
${Boost_SYSTEM_LIBRARY}
PythonEval
# ${ATOMSPACE_PythonEval_LIBRARY}
# ${Boost_SYSTEM_LIBRARY}
${PYTHON_LIBRARIES}
${ATOMSPACE_PythonEval_LIBRARY}
boost_system
)
ENDIF(HAVE_CYTHON)
27 changes: 27 additions & 0 deletions atomspace/diary.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1921,3 +1921,30 @@ Feb 2020 compiler: gcc (Debian 8.3.0-6) 8.3.0

Notes: Expected to maybe see some kind of change in performance.
It seems to be miniscule, lost in the noise.

4 Aug 2020
-----------
Partial measurements. Fanny. One pull req of concern:

Pull req #2744 - Stub out signals. No one uses them.

./atomspace_bm -A -n 4000000

-A -A
test pull #2744 Feb 2020
name K-ops/sec K-ops/sec Comments
---- --------- --------- -----------------
noop 1.70e6 2.00e6 no change expected
getType 94K 96K "
getTV 4689 4535 see notes
setTV 1244 1163 "
pointerCast 11.0K 11.4K no change expected
getIncomingSet 3253 3332 "
getOutgoingSet 50.5K 49.8K "
addNode 711 701 see notes
addLink -A 189 183 "
addLink -m 217 206 "
removeAtom 1359 1213 "

Notes: Removing the signal callbacks should improve performance.
... and now we know how much. We'll go for 3% to 7%, depending...
8 changes: 7 additions & 1 deletion micro/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,14 @@
LIST(APPEND LIST_MODULES
benchmark.cc
pointercast_bm.cc
values_bm.cc
atomspace_bm.cc
addnode_bm.cc
addlink_bm.cc
evaluationlink_bm.cc
large_simple_bm.cc
large_flat_bm.cc
large_zipf_bm.cc
values_bm.cc
scopelink_bm.cc
variables_bm.cc
)
Expand Down
33 changes: 33 additions & 0 deletions micro/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,38 @@
# Micro-Benchmarking

## WARNING
**THIS BENCHMARK DOES NOT MEASURE WHAT YOU THINK IT DOES**
The "Google Benchmark" assumes that all calls to some function
are all exactly the same, and that there is no hysteresis or
side-effects. This means that this benchmark is not suitable
for measuring AtomSpace insertions, because these have both
hysteresis and side-effects. Thus, the reported times
***ARE JUST PLAIN WRONG!***

The issue is this: Say we want to measure how much time it takes
to add 1M Atoms to the atomspace, vs how much time it takes to
add 64K Atoms. Well, the actual timing loop does NOT actually
iterate 1M times, or 64K times. It iterates some other number of
times that is not predictable.

Thus, to measure 64K insertions, you have to count to 64K, and then
clear the atomspace... inside the timing loop! But we didn't want
to measure how long it takes to clear the atomspace! Worse, the loop
might run 150K times, so there are 150K == 64K+64K+22K adds, and
the short fill-up at the end skews everything. Worse still, if
you ask for 1M insertions, the loop maight only run to 400K, so you
never even got to the end of the fill!

As a result, the numbers generated by this benchmark are insane.
They might give you some kind of general impression for what's going
on, but they will be wrong by factors of 2x or 4x or 5x ... usually,
reporting performance that is much faster than what it actually is.

If you have a lot of free time on your hands, maybe you can read the
docs and figure out how to fix this thing and make it work right.
But right now, its fubar'ed and totally unusable.


## Prerequisites
This requires the "Google Benchmark" micro-benchmarking tool v1.3.0 or higher.

Expand Down
105 changes: 105 additions & 0 deletions micro/addlink_bm.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
/*
* addlink_bm.cc
*
* Copyright (C) 2020 OpenCog Foundation
* All Rights Reserved
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License v3 as
* published by the Free Software Foundation and including the exceptions
* at http://opencog.org/wiki/Licenses
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program; if not, write to:
* Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

#include <benchmark/benchmark.h>

#include <opencog/atoms/base/Node.h>
#include <opencog/atoms/base/Link.h>
#include <opencog/atomspace/AtomSpace.h>

#include "benchmark.h"

using namespace opencog;

static void BM_AddLink(benchmark::State& state)
{
const size_t num_to_add = state.range(0);
AtomSpace* as = new AtomSpace();

// 101 and 233 are prime numbers. Thus, the links will interconnect
// to form some kind of nice polytope.
const size_t number_of_atoms = state.range(0);
std::vector<Handle> atoms(number_of_atoms);
for (size_t i = 0; i < number_of_atoms; ++i)
atoms[i] = createLink(LIST_LINK,
createNode(CONCEPT_NODE, "barfology" + std::to_string(i%101)),
createNode(CONCEPT_NODE, "blingometry" + std::to_string(i%233)));

size_t i = 0;
size_t j = 0;
for (auto _ : state)
{
if (j%3 == 0) // we are creating 3 atoms per pop...
as->add_atom(atoms[i++ % number_of_atoms]);
j++;

if (num_to_add < j)
{
as->clear();
j = 0;
}
}
delete as;
}

// Cannot go higher than 17 because the benchmark doesn't
// iterate enough times.
BENCHMARK(BM_AddLink)->Arg(2<<9)->Arg(2<<16)->Arg(2<<17);

static void BM_CreateAddLink(benchmark::State& state)
{
const size_t num_to_add = state.range(0);
AtomSpace* as = new AtomSpace();

std::vector<std::string> aname(101);
std::vector<std::string> bname(233);
for (size_t i = 0; i < 101; ++i)
aname[i] = "barfology" + std::to_string(i);
for (size_t i = 0; i < 233; ++i)
bname[i] = "blingometry" + std::to_string(i);

size_t i = 0;
size_t j = 0;
for (auto _ : state)
{
if (j%3 == 0) // we are creating 3 atoms per pop...
{
// Make a copy so that move constructore works right.
as->add_link(LIST_LINK,
as->add_node(CONCEPT_NODE, std::string({aname[i % 101]})),
as->add_node(CONCEPT_NODE, std::string({bname[i % 233]})));
i++;
}
j++;

if (num_to_add < j)
{
as->clear();
j = 0;
}
}
delete as;
}

// Cannot go higher than 17 because the benchmark doesn't
// iterate enough times.
BENCHMARK(BM_CreateAddLink)->Arg(2<<9)->Arg(2<<16)->Arg(2<<17);
85 changes: 85 additions & 0 deletions micro/addnode_bm.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*
* addnode_bm.cc
*
* Copyright (C) 2020 OpenCog Foundation
* All Rights Reserved
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License v3 as
* published by the Free Software Foundation and including the exceptions
* at http://opencog.org/wiki/Licenses
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program; if not, write to:
* Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

#include <benchmark/benchmark.h>

#include <opencog/atoms/base/Node.h>
#include <opencog/atomspace/AtomSpace.h>

#include "benchmark.h"

using namespace opencog;

static void BM_AddNode(benchmark::State& state)
{
AtomSpace* as = new AtomSpace();

size_t seed = 0;
const size_t number_of_nodes = state.range(0);
std::vector<Handle> nodes(number_of_nodes);
for (size_t i = 0; i < number_of_nodes; ++i)
nodes[i] = createNode(CONCEPT_NODE, get_unique_name("barfology", seed));

size_t i = 0;
for (auto _ : state)
{
as->add_atom(nodes[i++]);
if (number_of_nodes <= i)
{
as->clear();
i = 0;
}
}
delete as;
}

// Cannot go higher than 17 because the benchmark doesn't
// iterate enough times.
BENCHMARK(BM_AddNode)->Arg(2<<9)->Arg(2<<16)->Arg(2<<17);

static void BM_CreateAddNode(benchmark::State& state)
{
AtomSpace* as = new AtomSpace();

size_t seed = 0;
const size_t number_of_names = state.range(0);
std::vector<std::string> names(number_of_names);
for (size_t i = 0; i < number_of_names; ++i)
names[i] = get_unique_name("barfology", seed);

size_t i = 0;
for (auto _ : state)
{
// Make a copy so that move constructore works right.
as->add_node(CONCEPT_NODE, std::string({names[i++]}));
if (number_of_names <= i)
{
as->clear();
i = 0;
}
}
delete as;
}

// Cannot go higher than 17 because the benchmark doesn't
// iterate enough times.
BENCHMARK(BM_CreateAddNode)->Arg(2<<9)->Arg(2<<16)->Arg(2<<17);
55 changes: 55 additions & 0 deletions micro/atomspace_bm.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* atomspace_bm.cc
*
* Copyright (C) 2020 OpenCog Foundation
* All Rights Reserved
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License v3 as
* published by the Free Software Foundation and including the exceptions
* at http://opencog.org/wiki/Licenses
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program; if not, write to:
* Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

#include <benchmark/benchmark.h>

#include <opencog/atoms/base/Node.h>
#include <opencog/atomspace/AtomSpace.h>

#include "benchmark.h"

using namespace opencog;

static void BM_AtomSpace_Basic(benchmark::State& state)
{
AtomSpace* as;

for (auto _ : state)
{
as = new AtomSpace();
delete as;
}
}
BENCHMARK(BM_AtomSpace_Basic);

static void BM_AtomSpace_OneNode(benchmark::State& state)
{
AtomSpace* as;

for (auto _ : state)
{
as = new AtomSpace();
as->add_node(CONCEPT_NODE, "foobariffic");
delete as;
}
}
BENCHMARK(BM_AtomSpace_OneNode);
Loading

0 comments on commit ddc8966

Please sign in to comment.