CatchProbe Intelligence Technologies
Senior Database Engineer(Elastic/Mongo/Hadoop) (San Francisco) at CatchProbe Int
CatchProbe Intelligence Technologies, San Francisco, California, United States, 94199
Overview
Senior Database Engineer(Elastic/Mongo/Hadoop)
– San Francisco, CA. Workplace Type: Remote - Region: San Francisco, CA. Job Description
Must have experience with MongoDB installations, upgrades, and support. Responsible for administration, maintenance, performance analysis, and capacity planning for MongoDB/ElasticSearch/Hadoop clusters. Responsibilities
Coordinate and plan with application teams on MongoDB capacity planning for new applications. Use MongoDB base tools such as mongodump, mongoexport, mongorestore, mongoimport, mongostat, mongotop. Write queries in the mongo shell and in JSON scripting; support sharded clusters and perform upgrades and configuration maintenance on sharded clusters. Address, monitor, and manage capacity requirements (CPU, memory, storage). Assist application teams with assessment/resolution of performance bottlenecks in the MongoDB tier. Be aware of authentication/authorization methods in MongoDB (SCRAM-SHA1, X509, LDAP) including TLS/SSL and/or LDAP reconfiguration. Develop automated solutions for ad-hoc script execution, ad-hoc report generation, upgrades, and installs. Experience with NoSQL technologies and ability to use Cloud Manager; share relevant metrics for deployments when issues arise. Experience with Docker: deploying Mongo containers and supporting MongoDB administration in containers. Knowledge of administration and support of Hadoop systems is a plus; deploy Hadoop (Big Data) clusters, manage nodes, track jobs, monitor services (Zookeeper, HBase, SOLR), configure NameNode HA, scheduling, backups & restores. Develop scripts to review logs and alert on long-running queries. Demonstrable expertise in data storage technologies (Postgres/MySQL, Kafka/Kinesis) and strong scripting in Python (preferred) and Shell. Build services and expose internal APIs for data infrastructure automation components. Required Skills
Strong understanding of relational and non-relational databases, with guidance on use-cases. 5+ years installing, automating, scaling, and supporting NoSQL databases (MongoDB, Elasticsearch, Hadoop) and related technologies. 1-2 years experience with databases in public clouds (Hetzner, AWS, Azure, GCP). Automation proficiency; knowledge of Ansible, Python, Terraform. Willingness to learn other database, automation, and cloud technologies. Excellent communication and collaboration skills; software development experience and modern development processes. Experience with programming languages such as Java, Go, Node.js; knowledge of HTML/CSS/Bootstrap is a plus. Knowledge or experience in AI/ML is a plus. Ability to multi-task, lead teams, and work well under pressure; independent judgment and discretion in significant matters. DataLake cluster ownership and technical contact for Hadoop applications; onboarding new applications to DataLake by reviewing requirements and design. Lead end-to-end issues resolution and provide domain expertise. Core Responsibilities
Develops solutions for complex systems engineering problems; creates documentation for systems engineering and architecture. Database platform management, replication, policies, standards, and documentation (availability, backups, change management, security, and audits). Storage and backup management, SAN software expertise, and policy creation. Scripting and development across modern languages; scalable designs with robust APIs for integration. Networking design and diagnostics; familiarity with TCP/IP and related protocols. Application technologies and web services guidance; scalability, design, cost, troubleshooting support. Audit monitors and alarms; define tools required for operations. Project management and clear communication; client or vendor liaison as needed. Consistent independent judgment in significant matters; regular attendance. Seniority level
Mid-Senior level Employment type
Full-time Job function
Information Technology Industries
Software Development Note: This job posting may include standard notices related to referrals. If you need to sign in to set job alerts, follow the prompts on the site.
#J-18808-Ljbffr
Senior Database Engineer(Elastic/Mongo/Hadoop)
– San Francisco, CA. Workplace Type: Remote - Region: San Francisco, CA. Job Description
Must have experience with MongoDB installations, upgrades, and support. Responsible for administration, maintenance, performance analysis, and capacity planning for MongoDB/ElasticSearch/Hadoop clusters. Responsibilities
Coordinate and plan with application teams on MongoDB capacity planning for new applications. Use MongoDB base tools such as mongodump, mongoexport, mongorestore, mongoimport, mongostat, mongotop. Write queries in the mongo shell and in JSON scripting; support sharded clusters and perform upgrades and configuration maintenance on sharded clusters. Address, monitor, and manage capacity requirements (CPU, memory, storage). Assist application teams with assessment/resolution of performance bottlenecks in the MongoDB tier. Be aware of authentication/authorization methods in MongoDB (SCRAM-SHA1, X509, LDAP) including TLS/SSL and/or LDAP reconfiguration. Develop automated solutions for ad-hoc script execution, ad-hoc report generation, upgrades, and installs. Experience with NoSQL technologies and ability to use Cloud Manager; share relevant metrics for deployments when issues arise. Experience with Docker: deploying Mongo containers and supporting MongoDB administration in containers. Knowledge of administration and support of Hadoop systems is a plus; deploy Hadoop (Big Data) clusters, manage nodes, track jobs, monitor services (Zookeeper, HBase, SOLR), configure NameNode HA, scheduling, backups & restores. Develop scripts to review logs and alert on long-running queries. Demonstrable expertise in data storage technologies (Postgres/MySQL, Kafka/Kinesis) and strong scripting in Python (preferred) and Shell. Build services and expose internal APIs for data infrastructure automation components. Required Skills
Strong understanding of relational and non-relational databases, with guidance on use-cases. 5+ years installing, automating, scaling, and supporting NoSQL databases (MongoDB, Elasticsearch, Hadoop) and related technologies. 1-2 years experience with databases in public clouds (Hetzner, AWS, Azure, GCP). Automation proficiency; knowledge of Ansible, Python, Terraform. Willingness to learn other database, automation, and cloud technologies. Excellent communication and collaboration skills; software development experience and modern development processes. Experience with programming languages such as Java, Go, Node.js; knowledge of HTML/CSS/Bootstrap is a plus. Knowledge or experience in AI/ML is a plus. Ability to multi-task, lead teams, and work well under pressure; independent judgment and discretion in significant matters. DataLake cluster ownership and technical contact for Hadoop applications; onboarding new applications to DataLake by reviewing requirements and design. Lead end-to-end issues resolution and provide domain expertise. Core Responsibilities
Develops solutions for complex systems engineering problems; creates documentation for systems engineering and architecture. Database platform management, replication, policies, standards, and documentation (availability, backups, change management, security, and audits). Storage and backup management, SAN software expertise, and policy creation. Scripting and development across modern languages; scalable designs with robust APIs for integration. Networking design and diagnostics; familiarity with TCP/IP and related protocols. Application technologies and web services guidance; scalability, design, cost, troubleshooting support. Audit monitors and alarms; define tools required for operations. Project management and clear communication; client or vendor liaison as needed. Consistent independent judgment in significant matters; regular attendance. Seniority level
Mid-Senior level Employment type
Full-time Job function
Information Technology Industries
Software Development Note: This job posting may include standard notices related to referrals. If you need to sign in to set job alerts, follow the prompts on the site.
#J-18808-Ljbffr