Engineering vs coding

C++
1

I am a trader getting into the algorithmic side of it all. And looking for more engineering resources. I have a decent grasp on coding logic to syntax in C++ but how do you get information from one spot to another. I know about API and some DTC protocol but need to learn how to get data from one spot to another and looking for information on how the pipeline of engineering works and having trouble finding recourses, as it mostly syntax logic. I understand how to code script to make me toast at time, how dark I want it, what kind of bread etc but how do I plug that into my toaster so to speak?

Any information would be appreciated

2

Is it about transferring data between classes or deploying an application?

It would help if you can specify what you want to happen in your code or your application.

3

I work mostly with financial data and math and use a data feed that uses a DTC protocol to get information in and out and then brokers who typically use API. I understand the logic of coding the algorithm to say buy google when the cash flow rate is at X amount and price is at a certain level and saying the S&P is doing something specific. The syntax of that script, I have a decent grasp on that but I am looking for how you actually get that information and how the script I code is communicating with say a data provider give me that information.

and not necessarily the exact how but more the learning logic of how coding actually becomes an actual application of what you have coded. There is a lot syntax logic but very little on how a thing actually becomes a real life functionality …like sure to your phone if you make and app. So you code an app…how it able to become a functionality to you phone…what the compiler doing relative to all the phones functionality. And so looking for more engineering learning resources over coding syntax logic.

Thanks for the response and taking the time!

4

I think that I understand what you are trying to say. If something is happening, then how is it happening. That is what you want to know.

For two entities to communicate, there must be a medium accessible by both of them. Here, first entity is a machine. But machines being machines use signals of 0s (low) and 1s (high) to generate output(s) from the provided input(s). This is called a low level language (machine language).
For more on low level languages, look here.

The second entity, a human, uses sounds and gestures in general to communicate. This is called a high level language. High level languages have more human readability while low level languages have more machine (less human) readability. The machine cannot understand our language symbols and we have difficulty memorizing 0s and 1s. Therefore a mediator is required.
For more on computer languages, look here.

The translation process is divided into multiple stages to make it easy to convert human preferred symbols into machine preferred symbols. The human writes symbols and tells the machine what they mean. A new common medium called a programming language is created.

A programming language uses a compiler (example C++) or an interpreter (example Python). Basic working is the same for both, that is to convert high level language to low level language.
For more on their differences, look here.

Now. Things get complex when many things get involved. Take your case for example:

  1. A company registers its stocks on any government stock exchange service. A record of all share holders is managed by the company including the dividend amount received, purchase and sell records including data on buyer and seller. This is generally done using a relational database management tool or a spreadsheet application.
    For more on relational database management system, look here.

  2. The stock market needs to display the buy and sell activity live. Transactions are all recorded and stored. This is still generally managed by database management tools as the stock values are being updated frequently and need to be accurate.
    For more on transaction, look here.

  3. The transactions must be available to all traders and they must be informed about the changes in stocks or shares as fast as possible. A client and server model is used here, where the clients are traders and the market is the server. All transactions, registrations or security are managed by the server. Any change made is common to all and is visible to all using data transfer protocols like HTTPS, TCP or any custom protocol.
    For more on client and server model, look here.
    For more on protocols, look here.

  4. Now that there is data, there may be information hidden (or not visible plainly) in it. Market trends are generally repeated and can be detected better using calculated changes in values. Data stored on the stock exchange is huge since it contains history of the stocks of a company since its registration. The data stored on the stock exchange can then be used to check for or develop algorithms to trade.
    For more on storing huge data (big data), look here.
    For more on data extraction (data mining), look here.

In this way, there are many things that are involved. Many tools and technologies are involved just to maintain a comfortable experience for the end user. This way businesses maintains its customer growth for a longer period of time.
For more on user experience or interface design, look here.

Coding is as basic to a computer scientist (computer engineer or whatever you prefer to call them) as swimming is to a fish. What creates the difference is the knowledge and variation in the application of ideas, usage of tools and finding better or efficient approaches.

If you are really interested in in-depth knowledge or working of computers, then you need to learn everything (a bitter truth for a computer scientist):

  1. Using a computer (Ctrl + C, V, P or working of RAM, CPU and GPU in general)

  2. Networking to connect multiple users (computers)
    2.1. Computer Networking
    2.2. Protocols
    2.3. Web applications
    2.4. Cloud technologies

  3. Working of a computer (what happens when you right-click → refresh on Windows)
    3.1. Operating Systems
    3.2. System software
    3.3. Application software
    3.4. Computer Architecture
    3.5. Theory of Computation
    3.6. Compiler Design

  4. Application development (using already developed algorithms or developing of new algorithms)
    4.1. Programming language(s)
    4.2. Data Structures and Algorithms
    4.3. Performing calculations and different tasks (using above two)

  5. Automating application usage (includes prediction and automating specific tasks)
    5.1. Artificial Intelligence and Machine Learning
    5.2. Data Science and Data Analysis (suited for your case)
    5.3. Natural Language Processing
    5.4. Image Processing

The list goes on, so I am ending it now. I hope I have answered all of your questions.

5

Yes a great deal of relevant and good information! And found some information on what I need. Thanks for taking the time to reply!

6

Well, it is essential for a trader entering algorithmic trading to comprehend the engineering and data flow aspects of connecting trading infrastructure components. This incorporates getting market information from information suppliers through APIs or conventions, handling and putting away the information, and laying out a solid and practical foundation for information transmission.
Thanks

7

Ive been trading for 16 years(successfully!) and algorithmic trading for 8. But I have always worked on the analytics/ math and research side and just passed things to engineers and they passed it back but have recently started my own firm and tried to hire engineers and after a ridiculous amount of time and money! I am just going to teach myself to engineer and do it myself. I have a data feed and compile the math to the algo already and I have a broker . The issue I am having is my algo is very different than the market norm. This is a formula I have been building for 16 years…hence the start my own shop! The data feed must be coded in C++ and has a DTC protocol and the broker is an API and there is not a lot of information on how to go DTC to API. And the second you go algo trading people jump over to python and a dynamic cast(they are all wrong!)

Yahoo stock data! Please! lol Information that can be used to actually make a living from. It’s not on the internet and it certainly isnt free! And if all this garbage on the internet was how you made money trading…then 95% of all traders would not lose all their money within 3 months start! Thats a math fact! 1% of all traders make a living trading and only 5% make money! And so if you look online for an answer to how you trade. Only 1% is actually the real answer! The question is…if you have a 100 people telling you the way. 1 of them is correct! Which one! And what I am finding with the engineering, is the same thing!!

There is so much money to be made at the speculation of how. That everyone gives you the answer to how because people want that answer in hopes they will be able to do it, so they pay the money and that saturates the market with all this garbage on this is how you do it! Public information is a sham! In fact the trading arena as a, is a sham! And it’s rooted in everyone can tell you how to do it! Relevant data. Real information can cost 200,000 a month to look at! Even then. You’re talking about 100s of millions of dollars that move in milliseconds, processing it is another game! I assure nothing you need is public!

The last place I worked the algo had 17,000 lines of code in five languages to 4 portal! Those four places had information coming in from 4 different continents and 7 countries! Unfortunately those tech guys keep their code just as secrete as “we” the formula guys kept the math hidden! We always handed those guys bad math and fixed it after we got the code, because we didnt want them having the information!

And my point to my rant! If Anyone our there is thinking of trading! Do not do it! ITs a trick! Banks are fully aware that the common person is willing to dump money and all this stuff on the Internet is a hoax! Trust me…guys with the answer got that answer with 20 years of doing it. They are a secrete hand shaking crew that looks at business folks with masters from Harvard and laugh at them! Because they think their education will help them and it ends up being useless! They can tell you everything about how and the economics but they know nothing about making money! IF they did! They certainly would not need a job! lol

having said all that> I am defiantly making some progress with all the videos in this site but having a little trouble finding information on how a DTC protocol to data works to an API protocol. Because it would take me two years to rebuild all that math on another data feed! And brokers use API. And my data feed does not want you using brokers outside their firm…they want you to use pre built api broker(which they have and why they go DTC and not API) But I am not able to use those brokers from my own shop. I have to use a different broker and so I am trying to find information on this process.

8

API vs Protocol.

9

Thank you so much! I have been asking this question for like a year and trying to understanding this information! This is exactly what I was looking for.

There are so few resources on the engineering logic and I work in such a cutting edge field. Readily available approaches do not work in the scope we work and so you have to both innovate and create methods in unconventional ways…so much information is tied to answers in a very specific way and nine times out of ten for us…we have to build those answers! In our field you have five guys in a room, all with very specific skills and I made the mistake of passing information to engineers and they would pass it back. So now I have a very advanced capacity but I am a newb at best when it comes to engineering and trying to navigate my way to learn very advanced methodology on the engineering side, that most times 3 guys with phds built…I dont even know the right questions to ask but this is exactly what I was trying to comprehend…so thank you!

11

I can understand what you are trying to say.

If you want to implement your ideas without any interference from others, then you can go for in-depth courses available online on platforms like Udacity or Coursera. You can also search any specific topic course. One which suits you.

12

Engineering is defined as the branch of science and technology concerned with the design, building, and use of engines, machines, and structures.

In the Computer Science context, this is data structures, paradigms, software architecture and so on and so forth (one would agree that certain related stuff like ML algorithm creation and optimization belong primarily to the Mathematical Realm and not to CS).

Coding is, in this context, the ability to materialize these concepts.

For the sake of answering the question on a way that can lead to knowledge increase, I assume, by reading the comments of the OP that the main concern is “how things behave under the hood and why is this working”.

The Gobblydigook

To answer that we need to understand how a transistor (a.k.a. transfer resistor) works so you can understand how digital computers work (by using zeroes and ones), then understand binary and how can you encode information in binary, then you reach C++, code a program and compile it.
You’ll receive binary code so gap fulfilled. Now if you want to understand how a C++ compiler transforms our code to binary code (parse trees and so), but now how the program runs? This opens the door to Random Access Memory, Data Storage etc to understand which are the steps on computing something and why.

Then maybe you reach JS to create a UI and some backend services in Node to bring something to life while using interOp with some C++ program for a critical building block of your software architecture.

This requires knowing how V8 engine works, whats the JIT compiler, how does it does its thing, what are the OSI model to understand how internet layers work, how JS runtime interacts with Cpp executable and so on and so forth.

Useful information:

You can keep on going for ages, so to avoid falling into an endless spiral, the first thing you should do is going to the fundamentals.
There’s a book called Computer Science, an overview by J Glenn Brookshear which I recommend to anyone that wants to become a professional on CS.

It will give you the required surface knowledge regarding the big picture of CS which you can then explore by knowing the concepts (it’s way easier to find details of something if you already know that it exists) plus the book provides a good amount of information that helps on connecting the dots an understanding the whole much better.

Best regards

13

Thanks Joe!! I’ll check that book out! There is such a gap in information these days. My dad built race car engines growing up and he’d go…look, either you know speed or you don’t! And a million people will go, speed is mass, acceleration and rate and tell you everything about it! But so few humans can make a car go fast! And he always told me… you have to start with the logic from the top down to how something is functioning if you are going to grasp it.

You’re so correct too. Knowing the right things to look for is critical. Most times I spend days looking for how to do something because I don’t know what I am looking for and its miles of answers.

The biggest gap I have is how it’s all tied together as a whole and then how that whole is from my computer to say a bank in china thats using this functionality in this way…and I know there is an answer on this exact statement. The coding all makes sense. Code logic anyway. But I want to learn more about the complexity of the relationship of it all and what’s happening when it occurs…where it goes. how’s it goes and how’s it received, how its stored , where its stored …from an end to end perspective. (if that makes sense)In the logic of the nature of how that pipeline is built and not the specifics on say API. More of look it doesn’t matter what your approach this is everything that needs to take place!

I’ll give that book a read! thanks!!

14

Sorry Joel, not Joe .

15

Hi Phil, I understand your PoV but you need to accept that some parts you mentioned are and will be black boxes. You won’t know exactly how a software product works behind the scenes unless the company explains that to the public, which most do at a higher level but near zero do it on detail (and for good).

When you remove the black boxes from the equation and stick to what you can “see”, most of it will be simply “messaging” between clients and services so you can focus on networking details but anyway, check the book I recommended to you, you may find it quite useful and it will make you do these much needed brain connections between different concepts :slight_smile:

Best regards

16

You can try coding with MQL5 coding language it is tailor made for trading.

if it helps
https://www.google.com/search?q=mql5&oq=mql5&aqs=chrome..69i57j0i512l8.4221j0j15&sourceid=chrome&ie=UTF-8