Merge #12324: speed up Unserialize_impl for prevector

86b47fa741 speed up Unserialize_impl for prevector (Akio Nakamura)

Pull request description:

  The unserializer for prevector uses `resize()` for reserve the area, but it's prefer to use `reserve()` because `resize()` have overhead to call its constructor many times.

  However, `reserve()` does not change the value of `_size` (a private member of prevector).

  This PR make the logic of read from stream to callback function, and prevector handles initilizing new values with that call-back and ajust the value of `_size`.

  The changes are as follows:
  1. prevector.h
  Add a public member function named 'append'.
  This function has 2 params, number of elemenst to append and call-back function that initilizing new appended values.

  2. serialize.h
  In the following two function:
  - `Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)`
  - `Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)`
  Make a callback function from each original logic of reading values from stream, and call prevector's `append()`.

  3. test/prevector_tests.cpp
  Add a test for `append()`.

  ## A benchmark result is following:
  [Machine]
  MacBook Pro (macOS 10.13.3/i7 2.2GHz/mem 16GB/SSD)

  [result]
  DeserializeAndCheckBlockTest  => 22% faster
  DeserializeBlockTest => 29% faster

  [before PR]
      # Benchmark, evals, iterations, total, min, max, median
      DeserializeAndCheckBlockTest, 60, 160, 94.4901, 0.0094644, 0.0104715, 0.0098339
      DeserializeBlockTest, 60, 130, 65.0964, 0.00800362, 0.00895134, 0.00824187

  [After PR]
      # Benchmark, evals, iterations, total, min, max, median
      DeserializeAndCheckBlockTest, 60, 160, 77.1597, 0.00767013, 0.00858959, 0.00805757
      DeserializeBlockTest, 60, 130, 49.9443, 0.00613926, 0.00691187, 0.00635527

ACKs for top commit:
  laanwj:
    utACK 86b47fa741

Tree-SHA512: 62ea121ccd45a306fefc67485a1b03a853435af762607dae2426a87b15a3033d802c8556e1923727ddd1023a1837d0e5f6720c2c77b38196907e750e15fbb902
This commit is contained in:
Wladimir J. van der Laan 2019-06-18 16:52:11 +02:00
commit 8777a80706
No known key found for this signature in database
GPG key ID: 1E4AED62986CD25D
3 changed files with 46 additions and 3 deletions

View file

@ -378,6 +378,21 @@ public:
fill(ptr, first, last); fill(ptr, first, last);
} }
inline void resize_uninitialized(size_type new_size) {
// resize_uninitialized changes the size of the prevector but does not initialize it.
// If size < new_size, the added elements must be initialized explicitly.
if (capacity() < new_size) {
change_capacity(new_size);
_size += new_size - size();
return;
}
if (new_size < size()) {
erase(item_ptr(new_size), end());
} else {
_size += new_size - size();
}
}
iterator erase(iterator pos) { iterator erase(iterator pos) {
return erase(pos, pos + 1); return erase(pos, pos + 1);
} }

View file

@ -670,7 +670,7 @@ void Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)
while (i < nSize) while (i < nSize)
{ {
unsigned int blk = std::min(nSize - i, (unsigned int)(1 + 4999999 / sizeof(T))); unsigned int blk = std::min(nSize - i, (unsigned int)(1 + 4999999 / sizeof(T)));
v.resize(i + blk); v.resize_uninitialized(i + blk);
is.read((char*)&v[i], blk * sizeof(T)); is.read((char*)&v[i], blk * sizeof(T));
i += blk; i += blk;
} }
@ -688,8 +688,8 @@ void Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)
nMid += 5000000 / sizeof(T); nMid += 5000000 / sizeof(T);
if (nMid > nSize) if (nMid > nSize)
nMid = nSize; nMid = nSize;
v.resize(nMid); v.resize_uninitialized(nMid);
for (; i < nMid; i++) for (; i < nMid; ++i)
Unserialize(is, v[i]); Unserialize(is, v[i]);
} }
} }

View file

@ -183,6 +183,26 @@ public:
pre_vector = pre_vector_alt; pre_vector = pre_vector_alt;
} }
void resize_uninitialized(realtype values) {
size_t r = values.size();
size_t s = real_vector.size() / 2;
if (real_vector.capacity() < s + r) {
real_vector.reserve(s + r);
}
real_vector.resize(s);
pre_vector.resize_uninitialized(s);
for (auto v : values) {
real_vector.push_back(v);
}
auto p = pre_vector.size();
pre_vector.resize_uninitialized(p + r);
for (auto v : values) {
pre_vector[p] = v;
++p;
}
test();
}
~prevector_tester() { ~prevector_tester() {
BOOST_CHECK_MESSAGE(passed, "insecure_rand: " + rand_seed.ToString()); BOOST_CHECK_MESSAGE(passed, "insecure_rand: " + rand_seed.ToString());
} }
@ -260,6 +280,14 @@ BOOST_AUTO_TEST_CASE(PrevectorTestInt)
if (InsecureRandBits(5) == 18) { if (InsecureRandBits(5) == 18) {
test.move(); test.move();
} }
if (InsecureRandBits(5) == 19) {
unsigned int num = 1 + (InsecureRandBits(4));
std::vector<int> values(num);
for (auto &v : values) {
v = InsecureRand32();
}
test.resize_uninitialized(values);
}
} }
} }
} }