Uncategorized

Setup npm proxy in a corporate network

It’s painful to work behind a corporate network and everything does not work out of the box. Simple command like npm install would not work and here is how to work through the proxy:

Assuming you somehow get nodejs installed in your corporate laptop, open the .npmrc file which typical in the path for windows C:Users<your_user_id>.npmrc or mac Users/<your_user_id>/npmrc

Edit the file with this:

https-proxy=http://yourcompanyproxy.com:80
proxy=http://yourcompanyproxy.com:80
strict-ssl=false
registry=http://registry.npmjs.org

Try to run npm install again it should work now!

One more tip if you have some dependency hosted in your corperate internal Nexus npm repository, for example in the @ npmcorp scope, then run the below command to nominate the correct registry URL:

npm config set @npmcorp:registry https://your-company-nexus:80/nexus/content/repository/npm-internal

Then you should be able to overcome dependency not found errors. Try it out 🙂

My MBA study trip to Germany


The key personal learning from the Germany trip is anti-fragile. Travelling involves many uncertainty and randomness. Despite my flight was delayed in Helsinki at Finland and I was suffering from the stressful presentation with jet lag, the journey is worth it as I learned a lot beyond the classroom for my career in terms of management skills. The trip was full of insightful and informative lectures, with topics ranging from macroeconomy of Germany, European Unions, hidden champions, industry 4.0 and artificial intelligence. Our visit to deep tech venture, Wattx, and Daimler AG factory with BMW cars enlightened me. The history of Germany used to be fragile during the world war, but the prosperity of the countries these days is anti-fragile.


We learned from the trip that Germany has the largest number of hidden champions, which are either top three of the world or number one on the continent, yearly revenue over 5 million euros and low level of public awareness. The economy of Germany is mainly based on export as well as characteristics with many Small to Medium Enterprise (SME) that are specialized in deep technology. Among the 1307 hidden champions, these family businesses are clustered and decentralized in different areas in Germany instead of all focusing in Berlin.


In my opinion, this start-up scene and decentralization in Germany makes the country anti-fragile. Big corporations are robust with hierarchical structure and many rules to monitor staffs, but it is fragile with a higher turn over rate and slower adaption to change in the market with less flexibility. Meanwhile, SMEs have a lean structure to drive higher performance culture, higher staffs engagement and more people-oriented. It is anti-fragile for business to have staffs to be able to multifunctionality with more frequently transfer of personnel between functions than in large business. The SMEs’ employee can also have more regular customer contact than in large corporations, so they can have a better understand of the customers and react to the market changes more quickly. It is particularly important for the SMEs to treat the employee as a human instead of regarding as just a replaceable piece of resource in big cooperations. The design of a lean structure and decentralization system makes the business anti-fragile and with long term perspective in mind to keep the best talent.


Something that is important for us to learn is the way these hidden champion in Germany are able to survive in the market with resilience. They are focused in particular segments and do one thing better than others in the niche market. They have a global reach, such as the early entrance of the Asia countries and China market. They have a deep portfolio with no outsourcing of core competence and produce all parts with quality up to their high standards. They have a positioning of premium quality products, such as expensive Yachts. Last but not least, they have continuous innovation driven by customers and top managers. As a result of a high proportion of revenue spending in Research and Development (R&D), they have a higher amount of patients per thousands employee compared to large corporate. All these factors are anti-fragile and make them thrive and grow even with the exposure of volatility and uncertainty in the global market.


As a reflection, I notice a difference in mindset between Germany and Hong Kong. In Hong Kong, most students have narrowed options to work in the same kind of sales-oriented job in banking and finance industry, instead of choosing to study in science and engineering. In contracts, due to the historical and geographical reasons, Germany has more sectors to work on, such as steel, iron, machinery, chemical industry, locomotives, cars, and electronics. The young generation has more options like vocational training for technical skills other than going to university as the only preferable career path. The diversity of career choices, the craftsmanship in manufacturing and innovation in technology makes the countries less fragile to risks.


Germany has economic stability with a low inflation rate, stable growth, trade surplus, and high employment. This is particularly important not to repeat the same mistakes for the economic crisis, great depression, and hyperinflation, contributing to the rise of Nazi in Germany with political instability during World War II and genocide. It is hard to imagine history before transformation to nowadays with social and political peace, democracy, no extremism, and weak military power. History matters for us to understand where do we come from, understanding the present and future challenges in order to have better risk management skills in chaos.


For me, one of the main reasons to study MBA is to seek an answer for my career in terms of management skills in the global economy. I was fascinated by how the huge gap between different countries’ wealth level. For example, after my study trip, I was travelling to Switzerland for leisure with the most expensive McDonald’s big mac meal I ever eat. It cost me 12 Swiss Franc, which is about $93 Hong Kong dollar! In Germany, the exact same quality big mac meal costs 7 Euro, which is about $62 Hong Kong dollar. Meanwhile, the world seems really unfair if you were born in a poorer economy within same Eurozone, such as Romania rural poverty with people suffering in horrors from human trafficking. During my visit to Berlin, Hamburg, Munich, and Frankfurt on this trip, I witness the success factors of Germany with anti-fragile.


In my career as in management level, I can apply what I learned to build an anti-fragile workplace. We need to build a company that is able to deal with randomness and uncertainty going beyond resilience or robustness in this digital changing world as we have seen in Germany. Small flat teams tend to be anti-fragile, large hierarchy structure tends to be fragile. Managers should worry about centralized power, leaders should give trusts to team members to solve complex problems and decision-making should be decentralized. Antifragile company is fostered in thinking that embrace experimenting yet avoids too-big-to-fail.


It is an exciting time to live in the age of artificial intelligence and automation. We witness our industry facing a fantastic mutation with the new frontier of unexplored opportunities, more connected hardware and more real-world data. I have gained the benefit with a different perspective from international experiences in Germany and continue to pursue my answer after graduating from my MBA study to be more anti-fragile in this rapidly changing world.


Case study: Li & Fung family business


In this article, I would discuss the lesson we can learn from this Li & Fung’s family. The insights we can get from this family to last over one thousand years is the culture to reinvent itself in order to survive. There are not many Chinese family businesses can last for so long successfully. Starting from 1906 in Guangzhou, Li & Fung was the first Chinese trader export company. Nowadays, it already expands in 40 different countries with over 20,000 staffs.


The succession of a family business is a big challenge. In Chinese, there is a saying that “Wealth could not last for over three generations.”. Even in the United States, according to a study of the family business by Brooklyn College, 70% of the family business cannot pass from the first generation to the second generation. 88% of the family business cannot pass to the third generation. Only 3% of the family business can pass to the fourth generation. However, Li & Fung’s family business is now already in the fourth generation as an exceptional case. Why it can break the rule? What are the success factors?


The answer lies in the global view and opened-mindset of the founders and successors so that the family business was able to ride on the big trend and wave in the macro-economy. For example, at the end of the Qing Dynasty, China was opened to trade. Li & Fung was able to be the first Chinese middle man, who is better than Westerners for Chinese goods. The founders had more opportunities to widen the horizon compared to other traders in mainland China through interactions with foreigners with better English language advantages. Then after World War II, the family business was about to grow together with Hong Kong as a rising manufacturer and clothes export. They also import ball pens and rename it to atomic pen (thanks to the atomic bomb to end the war) in order to make a good profit. Later on, due to the China reform, China became the factory of the world, Li & Fung was benefited from this trend, and the Chinese business contributed to more than 51% of total sourcing business.


However, in the fourth generation these days, it is facing a lot of troubles due to its failure to foresee the digital trend. It can be shown from the fact that Li & Fung refused to invest in Alibaba for three times. The family business was a dominant trade middleman for sourcing and they stuck in their comfort zone. They did not realise that it has a dead business model and start to made irrelevant by Alibaba. The third generation, Victor & William, were happy with a traditional and profitable business so that they have the inability to grasp the e-commerce and internet trend. Figures show that Li & Fung’s profit continues to decline and struggled to boost profit and revenue as global brands change the way they manage their supply chains with the decline of the middleman. From 6th March 2017, Li & Fung was removed from Hang Seng Constituent Stocks, signalling the big challenge of the family business.


Throughout history, it was not the first time for the family business to face challenges and they were able to survive by reinventing itself and pass-through this success factor to the next generations. During the Korean war, the United States had embargo to China, which greatly affected the Hong Kong re-export business of Chinese goods. The family business was able to change direction from focusing on re-export to export Hong Kong local goods in clothing.

During the early days, the family was a traditional family business without international corporate experience. The two brothers of the third generation were studying at Harvard business school in the United States and bring the knowledge of new modern management theory into the old traditional family business. They did not follow the old rules but improve the family business in a professional way. They brought high efficiency and robust system to the family business so that it was able to go public in the stock market, due to their initiation to separate ownership and business management, as well as good governance.


Li & Fung was able to transit from One Boss / Employees system to the proper management hierarchy. The management was improved from family members to professional managers with university degrees. The return of Hong Kong to China during 1997 was also a historical crisis of family business since many Chinese family businesses were immigrant to oversea countries. However, they were still able to survive with the mindset to change and ride on the wave of macro-economy.

Nowadays the fourth generation, Spencer Fung had also success this important family values. He also studied aboard with master in MBA from the United States. He is entrepreneurial with startup experience to co-found an online marketing platform in Silicon Valley. Even with the threat of trade war between the United States and China on tariffs with a lot of small Chinese factories shut down, the family business was able to find to diversity itself out of China to Vietnam and other Asia Pacific countries.

Furthermore, the next big trend these days is on speed, innovation and digitalization. The fourth-generation was able to join as a latecomer with strategic investments to leverage new technologies, reduce the lead time of supply chain and rapid prototyping with 3D virtual design. Li & Fung also invest in advanced analytics to improve speed, cost, lead time and traceability. They have predictive analytics to enhance business performance.


Overall, the family business was successful with the merging of Western modernisation in management and technology with Eastern wisdom in relationship and family values. They have revolutionary ideas to implement change and constant improvement. The family business has a Western system but also Eastern human touch management style. In the United States, efficiency is the number one priority so that the employer-employee relationship is simply based on performance. However, Chinese people have the wisdom to treat human not as a machine. The emotional side with Chinese elements gives staffs a sense of belongings while staying professional. This is critical for the family succession and deals with the challenges faced now by the fourth generation.

MiFID II — What are the impact and opportunities for investors?

MiFID II is designed to bring greater transparency and investor protection to the financial markets. It introduces more protection for all investor types from retail to professional and extends the types of financial instruments in scope.

Impact beyond Europe

Markets directly impacted include France, Greece, Malta and the UK. Although it is a law in the Euro, the impact of the new rules is far-reaching and effective worldwide. MiFID II is applicable across the European Union (EU) and has relevance for countries in the European Economic Area (EEA) which are not in the EU. This means any firm across the EEA that conducts investment activities or services in financial instruments will be subjected to the new rule.

The regulation will have an indirect impact beyond Europe, such as APAC. MiFID II does not directly apply to non-EEA firms, but it is relevant primarily where there are employees assisting in the origination of EEA underwritten MiFID II products and services, for example, debt and equity insurance.

MiFID II does not generally have an extraterritorial effect, but

  1. It would apply if a non-EEA entity were carrying out MiFID investment services in respect of FiMID products in the EEA
  2. Specific provisions have express extraterritorial effect, e.g. transaction reporting requirements apply to all branches of EEA entities

There are four main areas of impacts:

  1. Research — it is considered an inducement under MiFID II and therefore receipt of free research is banned.
  2. Reporting — Transaction reporting is to include additional trade details, including cost elements. All periodic statements need to be quarterly at a minimum. An investor will be notified when there is a 10% depreciation of a discretionary portfolio, FX Operations or FX forwarded contact.


  1. Best execution — investors will now have visibility of the quality of execution received and the venues used.
  2. Manufacturing and distribution — investors will need to be assigned Target Market attributes. Products will also need to be analyzed for Target Market to ensure not selling products to negative Target Market investors.

1. Research Unbundling

MiFID II adds a further restriction for portfolio managers and independent investment advisers. Research is permitted provided it is paid for separately from execution costs. They can only receive research and sales if both are paid for either

  • Directly out of their own resources or
  • Through a client funded Research Payment Account (RPA)

Therefore, the fee of analysis report service would constitute the transaction fee, which would be paid separately from execution. The research and execution costs must be disaggregated and disclosed with the agreement relating to the research payment clearly states the budget for research and other services. The new RPA-related agreements allocate research budgets in advance. The research and sales must be logged. This research unbundling prevents potential conflict of interest. Some bank analysts are looking very nervously at the new changes fearing their jobs are about to go. Others analyst at smaller or independent companies sees it as a chance to get pay fairly for the quality of their work.

Independent advisers and assets managers cannot receive inducements unless they are “minor non-monetary benefits”, or they pay for them. The list of “minor non-monetary benefits” is prescriptive and includes generic information and corporate issuer.

2. Reporting

MiFID II is introducing Quarterly reports in relation to holdings and discretionary portfolio management. New requirements for firms holding an account that includes leveraged financial instruments (LFIs) or other contingent liability transactions (CLTs) to report to the investor if the initial value of the instrument depreciates by 10%, and thereafter by further multiples of 10%..

LFIs and CLTs include instruments where investors are exposed to any actual or potential liability that exceeds the cost of acquiring the instrument or where exposure to the underlying risks may be magnified. This includes Options, Swaps / Forwards / Futures and Structured Products.

Boarder scope of the financial transaction now needs to be reported. Both counter-parties (investment firms) to trade must report transaction data (65 fields, which include investor details) to regulators. Reportable data includes information on the person or algorithm responsible for the investment decision and execution. Transaction Reporting may be done through an investment firm’s own arrangements, an Approved Reporting Mechanism (ARM) or by the trading venue through which the transaction was completed no later than the close of the following working day.

3.Best Execution

Best Execution means a regulatory duty for firms to take all enough steps to obtain the best possible result for clients, taking into account price, costs, speed, the likelihood of execution and settlement, size, nature or any other relevant consideration. Also, MiFID II requires venues to provide reports on the quality of execution, free of charge and downloadable in a machine-readable format. The best execution disclosure statement would include execution venues.

MiFID II introduces the new Organised Trading Facility (OTF) for non-equity instruments. It allows counter-parties to cross and the venue operator to exercise discretion. It is a multilateral system where multiple third-party buying and selling interest in bonds, structured finance products, emission allowance or derivatives are able to interact. All venues will be subject to pre- and post-trade transparency requirements.

Systematic internaliser (SI)s are investment firms that deals on own account when executing client orders on an organised, frequent, systematic and substantial basis outside of a trading venue. Specific threads hold has been introduced to determine whether a firm is an SI. The regime applies to all financial instruments.

For algorithmic and electronic trading, investors would have to evidence their ability and disclose the algorithms they are using and are required to have additional controls in place. They would need effective systems and risks controls in place with appropriate threshold and activity/risk limits.

4. Manufacturing and distribution

Investors should be categorised when they are on-boarded for the purposes of conducting investment activities or providing investment services, and this categorisation should be done a legal entity level. An investor categorisation will affect the regulatory obligations of firms. For example, you may be categorised as either An Eligible Counter-party (ECP), a Professional or a Retail investor. Local Authorities and small corporates that are retail by default can request to Opt-Up. Any parties transacting in a MiFID II product requires a Legal Entity Identifier (LEI).

MiFID II introduces the requirement for both manufacturers and distributions to identify a target market for the products they manufacture/distribute respectively. This analysis also needs to consider any negative target market, which is the population of customers that a product should not be sent to. Not all products are in the scope of MiFID II, for example, FX spot is out of scope.

Getting real-time data from Web to Excel

An undergraduate student asked me how to get real-time data from a website into an excel. The data is used for his homework assignment, but many financial data are not free of charge. Therefore I gave him a simple solution, which I am going to show you how as well:

First, open Microsoft excel 365 and navigate to the Data tab. There is an option to get data from other sources and we select Web as an example.


Next, we will input the URL which has the data you want to get from:


The navigator would then gives you the option with tables for you to choose from. We select the Table 8 here as an example:


Once this is done, the data is import to your excel already. However, right now it requires manual refresh. In order to auto refresh the data, we could right click on the query to change the properties with refresh every 1 minute.


This is doing pretty good already and performing an auto refresh. What if you are more aggressive and want nearly real-time data every second? Then you would need to write some code. Navigate to File -> Options -> Customize Ribbon, under the Main tabs, you can check and add the Developer tab.


Then at the Developer tab, select Visual Basic:


And select Insert -> Module, then copy and paste the code snippet below:

Sub refresh_data()
Sheets("Sheet2").Select
Range("B2").Select
Selection.Copy
Sheets("Sheet1").Select
Dim xCell As Range
For Each xCell In ActiveSheet.Columns(1).Cells
If Len(xCell) = 0 Then
xCell.Select
Exit For
End If
Next
ActiveSheet.Paste
Sheets("Sheet2").Select
ActiveWorkbook.RefreshAll
Application.OnTime DateAdd("s", 1, Now), "Refresh_data"
End Sub

Like this:


Click run and we are done! This code snippet can be translated into simple English: select sheet 2, copy the current value, go to sheet 1, select the last row of the column then paste the value, finally move to sheet 2 and trigger refresh all, finally trigger itself every second. We can now achieve our goal to get nearly real time data to excel.

Leave a comment below if you have any further questions 🙂

Enable HTTPS on AWS EC2 instance with Node.js and Nginx on Ubuntu server

I have an AWS EC2 instance running with NodeJS and Nginx on a Ubuntu 16.04 server. I am going to show you how to enable HTTPS instead of HTTP for your website using Let’s Encrpyt. HTTPS helps prevent intruders from tempering communication between your website and your users’ browsers. It is encrypted with Transport Layer Security (TLS) Certification. Let’s Encrypt is a certificate authority that provides free X.509 certificates.

First of all, ssh into your EC2 instance:

ssh -i <keyfile.pem> ubuntu@<public-ip-address>

Secondly, clone the Let’s encrypt repository into path /opt/letsencrypt

sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Thirdly, check if there is any application listening to port 80 by running:

netstat -na | grep ':80.*LISTEN'

Kill any process running if it returns. For example, if you already have nginx server running at port 80, then you may need to stop it via:

sudo systemctl stop nginx

Fourthly, cd /opt/letencrypt to your repository and then run to get the certificates:

./letsencrypt-auto certonly --standalone --email <your@email.com> -d <domain.com> -d <subdomain.domain.com>

If your encounter an error

OSError: Command /opt/eff.org/certbot/venv/bin/python2.7 - setuptools pkg_resources pip wheel failed with error code 1

Then export below before you run the script in the shell:

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"

Follow the instructions in the command prompt and you should now get your certs at the path /etc/letsencrypt/live/<domain.com>.

Fifth, config the Nginx config to redirect your HTTP traffic to HTTPS by edit the file:

sudo vi /etc/nginx/sites-available/default

And the content looks like this, remember to replace the <YourDomain.com> with your own one as well as the root path for your website:

server {
listen 443 ssl;
server_name <YourDomain.com>;
ssl_certificate /etc/letsencrypt/live/<YourDomain.com>/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/<YourDomain.com>/privkey.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
  root /var/www/yourPath
index index.html index.htm;
# Make site accessible from http://localhost/
server_name localhost;
  location / {
proxy_pass http://localhost:3000/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forward-Proto http;
proxy_set_header X-Nginx-Proxy true;
proxy_redirect off;
}
}
server {
listen 80;
server_name <domain.com>;
return 301 https://$host$request_uri;
}

You can test your config for any error by running command:

sudo nginx -t

If it is running okay, restart ngnix:

sudo service nginx stop
sudo service nginx start

Last but not the least, go to AWS console and make sure your security group has port 443 open for HTTPS:


Done! Go to your https of your domain to verify it is working. If you encounter something like 502 Bad Gateway, make sure your NodeJS application is running correctly and I am using PM2 to keep it alive. Let‘s make the internet more secure 🙂

Replace text in xml files with Powershell

Powershell Script

Yesterday I was working at client site. It is a Windows server isolated from external network. It is not allowed to install any third party software in the machine. However, I got a task in hand to replace all xml files name from “My Report” to “My Report (New)”. The original file temp.xml looks like this:

<ReportList>
<Report Name="My Report">
</Report>
</ReportList>

while the excepted output temp-new.xml file I want for import looks in a structure like this:

<ReportList>
<Report Name="My Report (New)">
</Report>
</ReportList>

I have no tools in hand and it would takes many hours to replace hundreds of files one by one manually. The only thing accessible for scripting is the PowerShell. Here are a few lines of code to get things done.

Step 1: Load all xml files inside my Test folder.

$files = Get-Childitem C:UsersvictorleungtwDesktopTest -Recurse -Include *.xml

Step 2: Modify all report name and add “ (New)” after the original name

$xmldata = [xml](Get-Content $file); 
$name = $xmldata.ReportList.Report.GetAttribute("Name");
$name = $name + " (New)"; $xmldata.ReportList.Report.SetAttribute("Name", $name);
$xmldata.Save($file)
}

Step 3: Change the file name from temp.xml to temp-new.xml

Get-ChildItem *.xml | Rename-Item -NewName { $_.name -Replace '.xml$','-new.xml' }

Done! All the files are changed. Happy coding 🙂