Table of contents
1.
Introduction
2.
Redis Search
2.1.
Features of Redis Search
3.
Working in an Isolated Situation
3.1.
Installation requirements
4.
Getting support
5.
Redis with RediSearch in use
5.1.
Execute tests
5.2.
Debugging
6.
Frequently Asked Questions
6.1.
How does Redis store data?
6.2.
Can Redis data be lost?
6.3.
How does RediSearch work?
6.4.
In what language Redis has been written?
6.5.
How many keys does Redis support?
7.
Conclusion
Last Updated: Mar 27, 2024

Developer Notes For RediSearch

Author Ankit Kumar
1 upvote

Introduction

Setting up the development environment (either Linux or macOS-based), constructing RediSearch, running tests and benchmarks, and troubleshooting both the RediSearch module and its difficulties are all elements of the RediSearch development process.

In this blog, we will learn Developer Notes for RediSearch, but before we move further we must have to understand RediSearch. 

Redis Search

RediSearch is a Redis module that adds query ability, secondary indexing, and full-text search to the database. Redis is an in-memory data structure store used as a database, cache, message broker, or streaming engine. Strings, hashes, lists, sets, sorted sets with range searches, bitmaps, hyperloglogs, geographic indexes, and streams are all available in Redis. Redis offers built-in replication, Lua scripting, LRU eviction, transactions, several levels of on-disk persistence, Redis Sentinel for high availability, and Redis Cluster for automatic division.

Redis uses an in-memory dataset to achieve higher performance. Redis may save your data in one of two ways, depending on your needs: occasionally dumping the dataset to a disc or adding each command to a disk-based log.

Asynchronous replication, rapid non-blocking synchronization, and auto-reconnection with partial resynchronization on the net split are all supported by Redis.

In the next section, we’ll see some features of RediSearch.

Features of Redis Search

There are many benefits of redis search some of them are listed below.

  • Suggestions for auto-complete.
  • Weights in the field.
  • Searches using prefixes.
  • Querying several fields.
  • Query clauses are optional.
  • Between sub-inquiries, Boolean queries with AND, OR, and NOT operators.
  • Indexing of numerous fields in a document in full text.
  • Additional indexing.
  • Complete document content retrieval or just their ids.
  • Conditional and partial document updates.
  • Document deletion and updating combined with trash collection from the index.

 

In the next section, we’ll see it's working in an isolated situation.

Working in an Isolated Situation

There are a number of advantages to developing in a separate environment, like keeping your workstation clean and developing for a different Linux distribution. A virtual machine (which is relatively easy to start using Vagrant) is the most generic solution for an isolated environment.

Docker is even more adaptable because it provides a relatively closer solution:

search=$(docker run -d -it -v $PWD:/build debian:bullseye bash)
docker exec -it $search bash

 

Then cd /build from within the container and continue as usual.

All installs contain within the Docker container in this manner. You may either re-invoke the container with the above docker exec or save the container's state to an image and re-invoke it at a later time:

docker commit $search redisearch1
docker stop $search
search=$(docker run -d -it -v $PWD:/build rediseatch1 bash)
docker exec -it $search bash

 

You can use any OS instead of debian:bullseye, with the host OS being the best option (so you can run the RediSearch binary on your host once it is built).

In the next section, we'll discuss the requirements for installation

Installation requirements

Depending on the study OS, numerous packages are required to build and test RediSearch. Ubuntu/Debian, CentOS, Fedora, and macOS are currently supported.

Enter the RediSearch directory first.

You can run if you have gnu make installed,

make setup

 

Consider the following as an alternative:

./deps/readies/bin/getpy2
./system-setup.py

 

system-setup.py will use the native package management and pip to install various packages on your system. It will automatically run sudo and ask for permission.

If you'd rather skip it, you can:

  • Examine system-setup.py and manually install packages.
  • To display installation commands without running them, use system-setup.py --nop
  • As previously said, use an isolated environment.
  • Because Python installations are known to be sensitive when not used in isolation, employ a Python virtual environment: python2 -m virtualenv venv; . ./venv/bin/activate

 

In the next section, we’ll see some useful commands.

Getting support

The following is a brief description of the development features provided by make help:

make setup         # install prerequisites (BEWARE: THIS WILL CHANGE YOUR SYSTEM)

make fetch         # Dependant modules should be downloaded and prepared.

make build         # assemble and link

  COORD=1|oss|rlec   # create coordinator (1|oss: Open Source, rlec: (Reliable Linux Environment) Enterprise)

  STATIC=1           # Construct a static library

  LITE=1             # construct RediSearchLight

  VECSIM_MARCH=arch  # VecSim build architecture

  DEBUG=1            # construct for debugging

  NO_TESTS=1         # turn off unit tests

  WHY=1              # describe CMake decisions(in /tmp/cmake-why)

  FORCE=1            # Make CMake run again (default)

  CMAKE_ARGS=...     # Additional CMake parameters

  VG=1               # construct for Valgrind

  SAN=type           # LLVM sanitizer is used to construct(type=address|memory|leak|thread) 

  SLOW=1             # Designs should not be multi-core. (for diagnostics)

make parsers       #  code for parsers

make clean         #Delete any construction artefacts

  ALL=1              # Delete the whole artefacts folder

make run           # use RediSearch to run Redis

  GDB=1              #use gdb to execute

make test          # Execute all tests (via ctest)

  COORD=1|oss|rlec   # coordinator of tests

  TEST=regex         # run regex-matching tests

  TESTDEBUG=1        #be extremely talkative (CTest-related)

  CTEST_ARG=...      #CTest should be given args.

  CTEST_PARALLEL=n   #carry out tests in parallel

make pytest        # Execute python tests (tests/pytests)

  COORD=1|oss|rlec   # coordinator of tests

  TEST=name          # e.g. TEST=test:testSearch

  RLTEST_ARGS=...    # args are passed to RLTest

  REJSON=1|0         # Load the RedisJSON module as well.

  REJSON_PATH=path   # 'path', use the RedisJSON module

  EXT=1              #(Existing) external environment

  GDB=1              # Realtime debugging with RLTest

  VG=1               # using Valgrind

  VG_LEAKS=0         # Valgrind should not be used to look for leaks.

  SAN=type           # LLVM sanitizer is used.(type=address|memory|leak|thread) 

ONLY_STABLE=1      # remove insecure tests

make c_tests       # Execute C tests (from tests/ctests)

make cpp_tests     # Execute C++ tests (from tests/cpptests)

  TEST=name          # e.g. TEST=FGCTest.testRemoveLastBlock

make callgrind     # generate a call graph

  REDIS_ARGS="args"

make pack          # make packages for installation

  COORD=rlec         # RLEC pack coordinator('redisearch' package)

  LITE=1             # RediSearchLight pack('redisearch-light' package)

make deploy        # packages to S3 copy

make release       # create a new version

make docs          # establish documentation

make deploy-docs   # distribute documentation

make platform      # construct for a specific platform

  OSNICK=nick        # basis on which to construct(default: host platform)

  TEST=1             # after the construction, execute tests

  PACK=1             # make a package

  ARTIFACTS=1        # artefacts to the host

make box           # create a container in /search using volumen mapping

  OSNICK=nick        # platform specifications

make sanbox        # CLang Sanitizer is use to make containers.

 

In the next section, we’ll see the running of Redis with RediSearch.

Redis with RediSearch in use

The commands below will start Redis and load the RediSearch module.

make run

 

To communicate with redis-cli, open it in another tab.

In the next section, we'll show how to Execute the Test in RediSearch.

Execute tests

Unit tests are divided into various categories:

  • make c_tests run the C tests in tests/ctests.
  • make cpp_tests runs C++ tests (activated by GTest), which are located in tests/cpptests.
  • make pytest runs Python tests (enabled by RLTest), found in tests/pytests.

 

make test can be used to run all tests. The TEST option can be used to run a single test, such as make test TEST=regex.

In the upcoming section, we'll learn the debugging process.

Debugging

Run make DEBUG=1 to construct for the purpose of debugging (allowing symbolic information and disabling optimization). To execute gdb, use the command make run DEBUG=1. In addition to standard methods for the setting debugger in gdb, the BB macro can be used to set a stop within the RediSearch code. Only when executing under gdb, will it have an effect.

 

This was all about Developer Notes For RediSearch. Let us now see some of the FAQs related to the topic.

Frequently Asked Questions

How does Redis store data?

Data is saved in memory because Redis is an in-memory database (or RAM). All data stored on a server is lost if it crashes. For the data on the disc, Redis has backup procedures in place. The data is loaded from the drive into memory when the server reboots.

Can Redis data be lost?

When the workload is high enough that the "partial sync" stops, Redis will lose data if a master shard is lost.

How does RediSearch work?

RediSearch is a progressive indexing strategy for fast index construction and deletion, allowing you to quickly create indexes on datasets (Hashes). The indexes allow you to do complex aggregations, filter by characteristics, numeric ranges, and geographical distance, and query your data at lightning speed.

In what language Redis has been written?

Redis has been written in C language.

How many keys does Redis support?

Redis can handle up to 2^32 keys per instance and has been tested to support at least 250 million keys. There are 2^32 elements in every hash, list, set, and sorted set. In other words, your system's memory is most likely your limit.

Conclusion

In this article, we have extensively discussed Developer Notes For RediSearch. We saw RediSearch Development and its works in an isolated situation, then its installation requirements and also some faqs related to this topic.

After reading about Developer Notes For RediSearch, are you not feeling excited to read/explore more articles on related topics? Don’t worry Coding ninjas has you covered. To learn to see  Redis ConfigRedis Administration Data types in RedisRedis SecurityKeyspace notifications in Redis

Refer to our Guided Path on Coding Ninjas Studio to upskill yourself in Data Structures and AlgorithmsCompetitive ProgrammingJavaScriptSystem Design, and many more! If you want to test your competency in coding, you may check out the mock test series and participate in the contests hosted on Coding Ninjas Studio! But if you have just started your learning process and are looking for questions asked by tech giants like Amazon, Microsoft, Uber, etc; you must look at the problemsinterview experiences, and interview bundle for placement preparations.

Nevertheless, you may consider our paid courses to give your career an edge over others!

Do upvote our blogs if you find them helpful and engaging!

Happy Learning!

Live masterclass